Cloudflare Logpush
Push a zone's HTTP requests dataset to Heading from Cloudflare Enterprise, with the token carried in the destination URL
Logpush sends a Cloudflare zone's request logs straight to an HTTP endpoint. No code, no Worker requests, and the volume is Cloudflare's problem rather than yours.
Official reference: Enable HTTP destination, HTTP requests log fields and Log output options.
Logpush is Enterprise only
Cloudflare lists Logpush as unavailable on the Free, Pro and Business plans. If you are on any of those, use the Cloudflare Worker recipe instead. It runs on every plan and reports the same data.
INGEST_HOST is a placeholder
INGEST_HOST stands in for the ingest hostname, which is not final yet. The setup dialog in Heading shows the real endpoint and property id. Copy them from there and replace INGEST_HOST everywhere it appears on this page.
The trap to read first
A Logpush job has one destination field, and that field is a URL. There is no box for request headers.
Cloudflare's way around this is that any header_* parameter in the destination URL is turned into a request header on the push. So the bearer token Heading gives you travels as a query parameter, and it has to be percent-encoded, because the space in Bearer YOUR_TOKEN is not legal in a URL and an unencoded one produces a header that authenticates nothing.
https://INGEST_HOST/api/ingest/crawl/PROPERTY_ID?header_Authorization=Bearer%20YOUR_TOKEN%20 is the space. Encode the token itself as well if it contains anything outside A-Z a-z 0-9 - _ . ~. Cloudflare percent-encodes special characters it finds unencoded during upload, so encoding it yourself is the only way to be sure of what arrives.
If your tooling makes that awkward, the endpoint also reads the token from a plain token parameter:
https://INGEST_HOST/api/ingest/crawl/PROPERTY_ID?token=YOUR_TOKENEither way the whole credential now lives in a URL that appears in your Logpush job configuration. Treat that configuration as a secret store.
Create the job
Open Logpush for the zone
In the Cloudflare dashboard, select the account, then the domain, then Analytics & Logs → Logpush. Select Create a Logpush job.
Choose the dataset
Select HTTP requests. This is the zone-scoped dataset of requests Cloudflare served, and it is the only one that carries what Heading needs.
Select the fields
There is no "select all" in Logpush, and a field you leave out never arrives. Select exactly these eight:
| Field | Used for |
|---|---|
EdgeStartTimestamp | When the request happened |
ClientRequestHost | The host check against the property's domain |
ClientRequestPath | The path. This field is the path without the query string, which is what Heading stores anyway |
ClientRequestMethod | Recorded with the request |
ClientRequestUserAgent | Crawler identification |
ClientRequestReferer | AI-referred entries |
EdgeResponseStatus | What Cloudflare served the crawler |
ClientIP | Crawler verification only. Checked against published operator ranges, then discarded |
Leave ClientRequestURI off. It carries the full query string, and there is no reason to send data that is going to be thrown away on arrival.
Set the output options
| Option | Value |
|---|---|
| Timestamp format | RFC3339 |
| Output type | ndjson, which is the Logpush default: one JSON record per line |
| Sample rate | Leave at 1.0, which is every record |
The dashboard defaults the timestamp format to rfc3339. The API defaults it to unixnano, so if you create the job through the API rather than the dashboard, set "timestamp_format": "rfc3339" in output_options explicitly.
Do not set a sample rate below 1.0. Heading counts crawler requests, and a sampled stream produces a number that looks plausible and is wrong, with no way to correct it later.
Add a filter only if the zone serves several tracked sites
Leave the filter empty in the normal case. Never filter on ClientRequestUserAgent: Heading filters server-side, and your human traffic is what produces the page inventory and every AI-referred entry.
If this zone serves several sites that you track as separate properties in Heading, create one job per property and filter each on ClientRequestHost. That is a host filter, which is the one filter that helps here.
Set the destination
Choose HTTP destination and paste the URL from the top of this page, with your property id and encoded token in it.
Cloudflare validates the destination before it saves the job, by sending a gzipped test.txt.gz containing {"content":"tests"}. Heading recognises that probe, answers it with a 200, and stores no crawl data from it — the probe carries no request, and an event invented from it would be a page view that never happened. It does appear in the delivery log, so a successful validation is visible in Heading as proof the destination was reached.
The probe still has to authenticate. If the job refuses to save, the validation is what failed, and the usual causes are a mistyped host, a token that did not survive encoding, or a property that is not on a paid plan.
Save, then watch Heading
Logpush pushes in batches on its own schedule, not per request, so the first delivery is minutes away rather than seconds. Open the setup dialog in Heading and wait for it.
What Cloudflare sends
Newline-delimited JSON, one record per line, gzip-compressed, in batches sized by Cloudflare's max_upload_bytes and max_upload_records settings. Field names are the ones you selected above, exactly as spelled. Heading decompresses the body on arrival, so there is nothing to configure about compression on either side.
What Heading stores
The same answer appears on every recipe page, because it is the question your client's security reviewer will ask.
| Data | Stored |
|---|---|
| Request path | Yes, with the query string stripped before anything is written |
| Host | Yes, and it has to match the property's domain |
| User agent | Yes. It is how a crawler is identified |
| Status code | Yes. What your origin served the crawler is half the value |
| Time of the request | Yes, as a UTC timestamp |
| Referer | Yes, matched against the AI platforms Heading already knows |
| IP address | No. Checked against the operator's published ranges at ingest, then discarded |
| Cookies | No. Never read, never stored |
| Request or response body | No. Never read, never stored |
| Query string | No. Stripped from the path on arrival |
Raw events are kept for a short window and pruned as new ones arrive. The daily counts built from them are what the page reads afterwards. Because no IP, cookie, query string or body is retained, connecting a log source adds no personal data to Heading and brings in no new subprocessor.
Check it worked
Setup is not finished when the job saves. It is finished when a real event lands.
Open Tracking → Outcomes, find the Crawl activity card, and select Set up. The delivery log there lists the attempts the endpoint has seen, accepted and rejected alike, and it updates within a few seconds of one arriving. Each row carries the response code, how many log lines were in the payload, how many of them matched, and, for a rejection, the reason and the fix. Two attempts leave no row at all: a request carrying no token, and a request sent to a property id that does not exist. An empty log under a sender you know is running means one of those two, so check the header and the URL before anything else.
| Status | Meaning |
|---|---|
| Not verified | Nothing has arrived yet. Setup is not finished |
| Connected | Batches are arriving and the most recent one was accepted |
| Rejecting | Batches are arriving and every one is being turned away. Nothing is being recorded |
| Nothing arriving | Batches arrived before and none has arrived for 24 hours. Check the sender at your edge is still pointed here |
| Disconnected | Someone switched ingestion off for this property in Heading. Anything still arriving is refused with a 403 and nothing from it is stored. Reconnect from the same dialog to start accepting events again |
An accepted delivery that stores nothing is normal. Most requests to a site are not crawler requests. What is worth looking at is a full day of deliveries where lines received is high and lines kept is zero.
Troubleshooting
The job will not save. Cloudflare tested the destination and the test failed. Check the hostname, and check that the URL you pasted is the one from the setup dialog rather than a truncated copy.
The job saved and nothing is arriving. Logpush batches, so a quiet zone can take a while to reach a push. Check the job is enabled on the Logpush page, and check the zone is the one actually serving the traffic.
Events arrive and all of them are rejected. A 401 is the token. The likeliest cause on this recipe is encoding: an unencoded space in Bearer YOUR_TOKEN produces a header that does not authenticate. Rebuild the destination URL with %20 and any other special characters percent-encoded. Tokens are also per property, so the property id in the path has to be the property the token belongs to, and a token rotated in Heading stops working immediately.
host_not_matched. A log line whose host is not the property's domain is dropped rather than counted as this property's traffic. The batch around it is still accepted, because the other lines in it may be yours, so this shows up as a note on an accepted delivery and never as a rejected one. A Cloudflare zone often serves more hostnames than the one you are tracking. The note counts the dropped lines and names the hosts they were for, up to three, most frequent first. Add a ClientRequestHost filter to the job, or connect the other hostname as its own property in Heading with its own endpoint and token.
Lines received is high, lines kept is zero, all day. Check that ClientRequestUserAgent is among the selected fields. Without it nothing can be identified as a crawler, and every line is dropped.