Guides / Squid

AI audit straight from access.log

Squid's access.log is a native input format for the audit: no export screen, no conversion, no vendor console. If you run the internet's oldest proxy, you are one grep away from a shadow AI inventory.

Layer: proxy
Identity: proxy-auth username, else client IP
Format: native access.log, as-is
Typical home: universities, labs, ops-minded orgs
Run a free audit on an access.log
shadow-ai-audit.pdf Cover of the sample Shadow AI audit report prepared for an example company, with its headline findings. Sample report View all 11 pages Opens in your browser · PDF
Native format

access.log needs zero preparation

The audit reads Squid's native log line directly. Rotate, slice by date, upload. That is the entire pipeline.

1789551132.482 212 10.40.8.77 TCP_TUNNEL/200 3921 CONNECT chatgpt.com:443 jkovacs HIER_DIRECT/104.18.7.90 - 1789551140.019 188 10.40.8.77 TCP_TUNNEL/200 5110 CONNECT huggingface.co:443 jkovacs HIER_DIRECT/108.157.214.100 - 1789551163.771 301 10.44.2.19 TCP_TUNNEL/200 2870 CONNECT quillbot.com:443 - HIER_DIRECT/104.19.11.34 - 1789551201.220 240 10.40.3.51 TCP_TUNNEL/200 4460 CONNECT api.elevenlabs.io:443 smehta HIER_DIRECT/34.120.208.123 -

What gets read per line

  • The CONNECT or GET target becomes the hostname; ports and paths are stripped.
  • The auth username field, when not "-", becomes the identity.
  • The client IP backs up lines with no username.

HTTPS is not a problem

  • CONNECT tunneling logs the destination host even though the payload stays encrypted.
  • No SSL bumping needed; the audit's evidence is exactly what CONNECT already records.

Custom logformat? Still fine

  • Custom formats parse as long as a hostname sits somewhere on the line.
  • Worst case, awk out timestamp, user and host into a CSV, which is also a native input.
Worked scenario

One university, two very different networks

A mid-size university proxies both its research clusters and its teaching labs through Squid, with proxy auth on staff and cluster accounts. The audit ran once per network. Sample data.

Research network: 26 tools

  • Model hubs, hosted inference APIs, agent frameworks and two hosted notebook platforms, mostly under named cluster accounts.
  • Verdict pattern: high sophistication, low policy risk appetite mismatch. Most rows land "allow with controls".
  • The real finding: two research groups piping datasets through a hosted inference API whose terms are silent on training. Grant-funded data deserves a written answer.

Teaching network: 19 tools

  • Chatbots, paraphrasers and homework tools across lab machines, many lines with no username, attributed per lab IP.
  • Verdict pattern: academic-integrity questions more than security ones, and one companion chatbot flagged abusive-purpose.
  • The real finding: paraphraser volume spikes align with assessment deadlines. That went to the teaching committee, not the firewall.

Two audits, two governance paths: research findings went to the data-stewardship office, teaching findings to the academic-integrity committee. Splitting the export by network is what made both conversations precise.

Slicing

Getting the right slice out of a busy proxy

Universities generate serious log volume. Three slicing habits keep uploads inside the caps and questions crisp.

Slice by network, not by campus

Separate uploads per subnet range answer separate questions, as the scenario showed. One merged file answers a blurrier question with bigger numbers.

Slice by date with rotation

Rotated logs make windows trivial: last month is a file glob. Concatenate rotations up to 2,000,000 lines or 25 MB per full report.

Pre-filter the tunnel lines

CONNECT lines carry the hostname evidence. Grepping to them alone cuts file size sharply without losing a single AI finding.

Uploads are read once and discarded. Usernames appear only in your report's per-user table, paths never appear anywhere, and reports are deletable before their 90-day expiry.

Identity grain

Proxy auth decides the table you get

Squid deployments mix authenticated and anonymous access more than any other source in this series.

Authenticated users

Staff and cluster accounts produce named rows: jkovacs, smehta. In research settings those map to groups and grants, which is the accountability that matters.

Anonymous lab traffic

"-" usernames fall back to client IPs, so teaching labs read per machine. For integrity questions, per-lab granularity is usually the right resolution anyway.

Mixed files are expected

Both shapes coexist in one upload and in one report. The per-source table simply lists usernames and IPs together, each with its tools and hits.

Output

What the report returns for a Squid export

Report sectionWith access.log
Summary tilesFull totals per uploaded slice: tools, high-risk, training exposure, abusive, unsanctioned.
Tool tableEvery matched host with category, risk, sovereignty and dated training verdicts, API subdomains folded in via walk-up.
Per-source tableAuth usernames and client IPs as sources with per-tool hit counts.
Sanctioned splitYour approved list, per network slice if you audit per network.
CSV + PDFFull-report tier: CSV for the data-stewardship office, PDF for the committee pack.
Ops notes

Squid-specific details that shape results

Interception mode loses usernames

  • Transparent interception cannot do proxy auth, so identity falls to IPs.
  • The inventory is unaffected; note the mode in your findings so readers expect IP grain.

Long-lived tunnels undercount

  • One CONNECT can carry hours of chat. Like DNS caching, treat hits as floors and lean on source counts.

Direct-to-internet leaks bypass the proxy

  • Anything with a route around Squid never logs. Egress rules that force the proxy keep the sensor complete.
  • A periodic DNS-source audit cross-checks the gap; the methodology page covers multi-source truth.

Old logs are an asset

  • Squid archives often reach back years. A retroactive audit of last spring's window turns policy debates into measured history.

Open the sample report before your first run

Ten pages of sample data show exactly what an access.log slice becomes: the exposure index, verdicts, per-source rows. Committee-ready without editing.

Open the sample PDF
Campus governance

Routing findings to the right committee

University shadow AI splits into three governance lanes. The report's columns tell you which lane each row belongs to.

Data stewardship

Training verdicts and sovereignty flags on research-network tools. Grant conditions and ethics approvals hinge on exactly these columns.

Academic integrity

Paraphrasers and homework tools on teaching networks, read as volume trends against the assessment calendar rather than as individual accusations.

IT security

Abusive-purpose flags and high-risk tools anywhere. The shortest list, the fastest action, usually a same-day block.

Semester-boundary re-runs per network keep all three lanes fed. Costs per report drop with packs, detailed on the pricing page.

Complementary sources

Pairing Squid with other log sources

Squid captures explicit proxy traffic. Pairing it with other sources closes coverage gaps without replacing the proxy.

DNS resolver logs

Pi-hole or NextDNS logs capture traffic that bypasses the proxy entirely: mobile devices, IoT endpoints and applications that ignore system proxy settings. Upload both sources in the same audit run for merged results.

Firewall connection logs

If a perimeter firewall sits behind or alongside Squid, its connection logs catch direct-connect traffic that evades the proxy. FortiGate, SonicWall and Palo Alto logs all upload alongside Squid exports.

Endpoint DNS

Cloudflare Gateway with WARP on laptops captures roaming traffic when users leave the campus network. Especially relevant for research staff working from home or conferences.

The audit merges and deduplicates across all uploaded sources automatically. No pre-processing needed. Each source adds coverage without creating duplicate findings.

Automation

Automating the quarterly Squid export

Two approaches to make the quarterly export routine instead of manual.

Cron-based rotation export

A simple cron job on the first of each quarter concatenates the last 30 days of rotated access.log files into a single export file. One scp or rsync to the upload workstation, then upload through the web interface.

Log aggregator export

If Squid logs already flow to a central syslog server, ELK stack or Splunk instance, export the hostname and identity fields for the audit window directly from the aggregator. Most export as CSV in under a minute.

Whichever method: save the export command as a runbook entry. The quarterly audit takes the same 15 minutes every time when the export is scripted rather than remembered.

FAQ

Squid export questions

Do I need to convert access.log before uploading?

No. The native format is supported directly, rotations concatenated and all. Custom logformats parse when a hostname is present on the line.

Does HTTPS tunneling hide AI tools from the audit?

No. CONNECT lines log the destination hostname, which is the exact evidence the matcher consumes. No SSL bumping required.

How should a university slice its logs?

Per network: research, teaching, halls, guest. Each slice becomes its own report, and each report lands with the committee that owns that network's question.

What about privacy review before uploading?

The upload contains hostnames, identities and timestamps; paths are dropped, the file is discarded post-run, reports delete on demand. That paragraph usually satisfies the review.

Can we audit historical semesters?

Yes, any archived rotation uploads like a current one. Retroactive baselines are the cheapest way to ground a policy debate in numbers.

What does this cost against a university budget?

Free previews for spot checks; $99 per full report, $60 each in the 5-pack. Two networks audited each semester is well under any committee's coffee budget. See pricing.

The one-liner kit

Three shell habits that make audits routine

Ops teams automate what they repeat. The whole export step reduces to muscle memory.

The monthly slice

A cron that gathers last month's rotations per subnet into dated files. Export day becomes "attach file", nothing more.

The CONNECT filter

Keeping only tunnel lines cuts most of the volume while preserving every hostname the audit can use. One grep, kept in the runbook.

The size check

A quick line count against the 2,000,000-line cap decides full-month upload versus two half-month runs, before anything leaves the box.

Teams that script the slice run audits on time. Teams that export by hand run them "next sprint". The scenario university automated after their second manual quarter.

grep, upload, inventory

Your access.log already knows which AI tools the campus reaches. Slice one network's month and see the free totals today.

Run the free audit