Netlify log drain
Forward a Netlify project's site traffic logs to Heading with no code, on the Netlify Enterprise plan
Netlify log drains send your site's traffic logs to any endpoint that accepts JSON or NDJSON. No code, and the drain runs whether or not your site uses functions.
Official reference: Log Drains.
Log drains are a Netlify Enterprise feature
Netlify lists log drains as available on the Enterprise plan only. On any other plan, use the Netlify edge function recipe. It reports the same data and runs on every plan that supports edge functions.
INGEST_HOST is a placeholder
INGEST_HOST stands in for the ingest hostname, which is not final yet. The setup dialog in Heading shows the real endpoint and property id. Copy them from there and replace INGEST_HOST everywhere it appears on this page.
Before you start
Get the endpoint and the bearer token from Heading. Open Tracking → Outcomes, find the Crawl activity card and select Set up. The token is derived from the property, so there is nothing per-property stored on our side. Treat it as a server-side credential.
Create the drain
Open log drains
In the Netlify dashboard, open the project, then Logs & Metrics → Log Drains.
Choose the General HTTP endpoint service
Netlify lists named services (Datadog, Splunk and others) and a generic option. Choose the General HTTP endpoint service.
Fill in the service settings
| Field | Value |
|---|---|
| Full URL | https://INGEST_HOST/api/ingest/crawl/{propertyId} |
| Authorization header | Bearer YOUR_TOKEN |
| Log Drain Format | NDJSON |
Enter the whole header value, including the word Bearer and one space before the token.
Select the log types
Select Site traffic logs. That is the one that carries visitor and crawler requests for your pages and assets.
Function, edge function, deploy and WAF logs can stay off. Heading has no use for them, and every line you send that it cannot read is bandwidth spent to be dropped.
Leave the PII exclusion off
Netlify offers an Exclude personally identifiable information (PII) option, which removes user_agent and client_ip from every line.
With user_agent removed there is no way to tell a crawler from a person, so no line can ever produce a crawl event. The connection still reads as active, because the same lines still build the page inventory — which is exactly what makes this failure hard to spot from the outside. Heading names it for you: a delivery in which no line carried a user agent says so on its own row in the delivery log.
Heading's own handling is on the next section of this page: the IP is checked against published operator ranges and discarded on arrival, and it is never written down.
Save it, then watch Heading
Netlify batches lines, so give the first delivery a few minutes on a quiet site. Open the setup dialog in Heading and wait for it.
What Netlify sends
Newline-delimited JSON, one traffic log object per line. Heading reads these fields:
| Netlify field | Used for |
|---|---|
timestamp | When the request happened |
url | The full request URL. Netlify sends no separate host field, so this is where both the host check and the path come from. The query string is stripped on arrival |
method | Recorded with the request |
status_code | What your site served |
user_agent | Crawler identification. Removed if you enable the PII exclusion |
referrer | AI-referred entries. Note Netlify's spelling, with two rs |
client_ip | Crawler verification only. Checked against published operator ranges, then discarded. Removed if you enable the PII exclusion |
log_type | Tells traffic lines apart from the other log types |
What Heading stores
The same answer appears on every recipe page, because it is the question your client's security reviewer will ask.
| Data | Stored |
|---|---|
| Request path | Yes, with the query string stripped before anything is written |
| Host | Yes, and it has to match the property's domain |
| User agent | Yes. It is how a crawler is identified |
| Status code | Yes. What your origin served the crawler is half the value |
| Time of the request | Yes, as a UTC timestamp |
| Referer | Yes, matched against the AI platforms Heading already knows |
| IP address | No. Checked against the operator's published ranges at ingest, then discarded |
| Cookies | No. Never read, never stored |
| Request or response body | No. Never read, never stored |
| Query string | No. Stripped from the path on arrival |
Raw events are kept for a short window and pruned as new ones arrive. The daily counts built from them are what the page reads afterwards. Because no IP, cookie, query string or body is retained, connecting a log source adds no personal data to Heading and brings in no new subprocessor.
Check it worked
Setup is not finished when you save the drain. It is finished when a real event lands.
Open Tracking → Outcomes, find the Crawl activity card, and select Set up. The delivery log there lists the attempts the endpoint has seen, accepted and rejected alike, and it updates within a few seconds of one arriving. Each row carries the response code, how many log lines were in the payload, how many of them matched, and, for a rejection, the reason and the fix. Two attempts leave no row at all: a request carrying no token, and a request sent to a property id that does not exist. An empty log under a sender you know is running means one of those two, so check the header and the URL before anything else.
| Status | Meaning |
|---|---|
| Not verified | Nothing has arrived yet. Setup is not finished |
| Connected | Batches are arriving and the most recent one was accepted |
| Rejecting | Batches are arriving and every one is being turned away. Nothing is being recorded |
| Nothing arriving | Batches arrived before and none has arrived for 24 hours. Check the sender at your edge is still pointed here |
| Disconnected | Someone switched ingestion off for this property in Heading. Anything still arriving is refused with a 403 and nothing from it is stored. Reconnect from the same dialog to start accepting events again |
An accepted delivery that stores nothing is normal. Most requests to a site are not crawler requests. What is worth looking at is a full day of deliveries where lines received is high and lines kept is zero.
Troubleshooting
Nothing is arriving. Check the drain is still listed and enabled on the Log Drains page, and that Site traffic logs is one of the selected types. Netlify only shows the base URL after you save, so if you suspect a typo in the path or property id, re-enter the full URL rather than trying to read it back.
Events arrive and all of them are rejected. A 401 is the token, and on this recipe the usual cause is a header value that lost its Bearer prefix. Re-enter the Authorization header as Bearer then a space then the token. Tokens are per property: the token for one property is rejected by another property's endpoint, and the property id in the URL has to be the property the token belongs to. Rotating the token in Heading invalidates the old one immediately.
host_not_matched. A log line whose host is not the property's domain is dropped rather than counted as this property's traffic. The batch around it is still accepted, because the other lines in it may be yours, so this shows up as a note on an accepted delivery and never as a rejected one. On Netlify this is usually deploy previews and branch deploys on *.netlify.app, or a second custom domain on the same project. The note counts the dropped lines and names the hosts they were for, up to three, most frequent first. Connect the other domain as its own property in Heading with its own endpoint and token if you want it counted.
Deliveries are accepted and no crawler ever appears. The PII exclusion is on. Turn it off: without user_agent there is nothing to identify a crawler with. The delivery log says so on the rows where it happened.