Troubleshooting and delivery health
Use this guide to diagnose delivery issues, confirm retries, and keep your webhook endpoints healthy. It covers quick checks, signature failures, idempotency, and recommended instrumentation.
Check the following
-
Confirm subscription
Use Get all subscriptions to confirm subscription (resourceFEEDS, eventSELLER_PERFORMANCE_NOTIFICATIONS, webhook endpoint URL). -
Reproduce in sandbox
Send a sample using Test notification. Refer to Test in sandbox. -
Inspect logs
Capture request headers and raw body bytes. Verify you are loggingWM_SEC.TIMESTAMP,WM_SEC.SIGNATURE, and a delivery identifier (for example,eventId). -
Return a 2xx only after a durable write
Acknowledge quickly after you store the event and enqueue any long work asynchronously.
Webhook failure notification
Walmart can email your account contacts if your subscription has repeated delivery failures. The email subject says Webhook Delivery Failure - Action Required. The table shows an aggregate of all failure reasons, events, and endpoints.
Recipient scope: At this time, webhook failure notifications are sent only to administrator email addresses on the account. To ensure the right people are alerted, add critical operational contacts (for example, on-call engineers or integration owners) as administrators.
Sample email
- Aggregated failure reason: Error in sending webhooks for deliveryId
- Error code: 500
- Message: Token not found
- Events: SELLER_PERFORMANCE_NOTIFICATIONS
- Endpoint URL: https://example.com/webhooks/walmart/performance
What to do
- Review logs for the window shown in the email.
- Verify that your webhook endpoint is active and accessible.
- Check your API credentials and network settings.
- Validate signature verification and seller checks.
- Fix the issue and redeploy if needed.
- Send a sandbox test delivery to confirm recovery.
Thresholds and email fields can vary. Your account may receive additional details when enabled.
Common symptoms and fixes
| Symptom | Likely cause | What to check | How to fix |
|---|---|---|---|
| No deliveries arriving | Disabled subscription, network/TLS block | endpointUrl in subscription, DNS, firewall/WAF, TLS cert chain, route availability | Update subscription URL, allow traffic from Walmart, confirm TLS, confirm route returns 2xx |
| Your service returns 400 | Schema validation is too strict | Envelope fields, required vs optional fields, unknown fields in payload | Accept the shared envelope and ignore unknown fields defensively; loosen validation to allow additive changes |
| Your service returns 401 | Signature validation failed | Auth middleware on the route, required headers, whether endpoint expects a token, signature validation path if you map failures to 401 | Remove unintended auth requirements for Walmart deliveries, or ensure validation is correct; return 2xx only after a durable write |
| Your service returns 403 | Seller authorization failed | WAF rules, IP allowlists, gateway policies, tenant/seller authorization logic, signature/seller mapping if you map failures to 403 | Update WAF/allowlist policies, fix seller mapping checks, ensure valid deliveries are not blocked |
| Your service returns 408 | Request timed out | Server timeout config, request duration, downstream calls during request, payload processing time | Acknowledge fast, move work to a queue, avoid downstream calls in the request path, tune server timeouts carefully |
| Your service returns 500 | Unhandled exception or dependency error | App logs, null/schema assumptions, DB/queue availability | Fix the exception, add defensive parsing, handle dependency failures, keep handler thin; return non-2xx only for failures you want retried |
| Your service returns 501 | Route or method not implemented | Correct path and HTTP method, API gateway route mapping, deployment version | Implement the POST handler, fix routing, redeploy, confirm it returns 2xx on success |
| Your service returns 502 | Bad gateway from a proxy or upstream | Load balancer/API gateway logs, upstream health, DNS, TLS handshake between gateway and service, upstream timeouts | Fix upstream availability, correct gateway routing, tune upstream timeouts if appropriate, ensure gateway can reach the service |
| Duplicate processing | Missing idempotency | Use of a delivery identifier on writes, retries after non-2xx/timeouts | Add a uniqueness constraint or conflict-safe upsert keyed by eventId or delivery id |
| Only one notification type arrives | Routing logic is filtering incorrectly | Filtering by payload.notificationType, routing rules, logging per type | Log payload.notificationType for every delivery, ensure both ALARMS and REPORT route to handlers, avoid hard-coded filters |
Signature verification failures
- Verify headers exactly as sent. Required headers include
WM_SEC.TIMESTAMPandWM_SEC.SIGNATURE(and optionallyWM_SEC.KEY_IDif you rotate secrets). - Validate the signature before processing the body.
- Use the exact request path and raw body bytes when computing the HMAC.
- Reject any request that fails verification and log the reason.
Refer to Security and authenticity.
Retries and backoff
- Walmart retries when your endpoint does not return 2xx.
- Each retry is signed. Always verify signatures on retries.
- Ensure your receiver is idempotent and safe to retry.
- When healthy, you should notice a single successful delivery with no subsequent retries.
Refer to Test in sandbox to validate retry behavior.
Idempotency and deduplication
- Use a delivery identifier (for example, a top-level
eventIdor a documented delivery header) as the idempotency key. - Make writes conflict-safe by using an upsert or a uniqueness constraint on the event identifier.
- Log both accepted and deduplicated events to prove correctness during audits.
Health monitoring and alerting
Track these key indicators:
- 2xx rate per endpoint and event type
- Retry rate and time to recovery
- Median and p95 acknowledgment latency
- Signature verification failure rate
Recommended alerts:
- 2xx rate below threshold for 5 minutes
- Retries above threshold for 10 minutes
- Signature failures above threshold for 5 minutes
Broken endpoints
If Walmart detects repeated failures, the platform can mark your endpoint as broken and notify your contacts. When that happens:
- Fix the root cause in your service.
- Validate with Test in sandbox.
- Resume normal traffic and watch error rates.
Sandbox tips
- Start with small payloads, then test larger alarm sets or metric arrays.
- Intentionally return a non-2xx once to observe retry cadence.
- Validate that your logs include headers and the raw body used for verification.
Next steps
- Verify your signature code and seller checks in Security and authenticity.
- Reproduce issues with Test in sandbox.
- Review payload shapes in the Event catalog.
Reference guide
This guide explains how to monitor delivery success, handle retries, and remediate failing endpoints.
- Get all subscriptions: Returns all subscriptions on your account for auditing and troubleshooting.
- Update subscription: Changes an existing subscription (endpoint URL, event set, or status).
- Delete subscription: Removes a subscription you no longer need.
- Test notification: Triggers a sample delivery to your callback URL to validate your integration.
Related
Updated 6 days ago
