Session Overflow and the Art of Not Storing Everything

The Starting Point

Simple request: add a "Description" column to a data import table. Each imported record has a transaction code — things like AC-4010.R, AC-4037.P — and the user wanted to see the human-readable description next to it. The descriptions already existed in a reference table in the database. Should've been a 20-minute job.

It turned into six commits, two production errors, one feature built-then-deleted-then-rebuilt, and a lesson in why you should never dump arrays into session flash data.

What Actually Happened

Round 1: The Static Lookup That Didn't Work

My first instinct was wrong. I built a massive static array on the model — 113 key-value pairs mapping every transaction code to its description. Hardcoded. In PHP.

public static function transactionDescriptions(): array
{
    return [
        'AC-4010.W' => 'Request New Record',
        'AC-4022.P' => 'Response to New Record',
        // ... 111 more entries
    ];
}

Deployed it. Descriptions didn't show. The column was there, but every row showed a dash.

The user sent me a screenshot showing the transaction types already existed in the database — there was a whole admin CRUD for managing them. The codes in the database (AC-4037.P, AC-4051.R) didn't match the ones in my static list because the static list came from a reference document, and the actual data used different suffixes.

I was solving the wrong problem. The data was already there; I just wasn't reading it.

Round 2: Use the Database

Ripped out the static lookup. Added a $typeDescriptions variable to both controller methods — a simple pluck('name', 'code') from the reference table. Updated both views to use $typeDescriptions[$record->transaction] ?? '—'. This worked immediately.

Then the user gave me an updated official reference document with 142 entries (the static list only had 113, and many had different descriptions). I wrote a tinker script to upsert all 142 into the database: 114 inserted, 23 updated, 5 unchanged. Also added a CSV import to the admin page so the user could do this themselves in future.

Round 3: The Import That Broke Production

Next request: "the last row of every import file is just 'Applied Filters' — skip it and don't show an error." Also: "show me which records were skipped as duplicates."

I added both features. The duplicate tracking stored every skipped record's details in an array, flashed the whole thing into the session, and rendered it in a modal.

Deployed. User tried to re-import the same file. 502 Gateway Timeout.

The production logs told the story:

upstream sent too big header while reading response header from upstream

Every single record in the file was a duplicate. The import_duplicates session flash contained an array of ~30 objects, each with four string fields. Serialised, this blew past nginx's default proxy_buffer_size for response headers.

Even with a database session driver, the session cookie itself plus other response headers were too large for nginx's upstream buffer. The PHP-FPM worker generated the response, but nginx killed it before the browser ever saw it.

Round 4: Cap It (Still Broken)

Quick fix: array_slice($result['duplicates'], 0, 50). Cap the session data at 50 entries.

Deployed. User tested again. The import completed — no timeout — but the duplicate feedback message didn't appear. The success bar just said "0 messages imported." with no mention of duplicates and no "View Skipped" button.

I spent a while tracing through the logic. The message-building code used $result['skipped'] - $duplicateCount to derive the error count, which was fragile. But that wasn't the main issue.

The main issue was that even 50 entries was still too much data in the session. The response headers were smaller but still being rejected by nginx on some requests. The user never saw the flash data because the response never arrived intact.

Round 5: Nuke It

I ripped out the entire duplicates modal. Removed the array from the session. Removed the modal HTML from both views. Just stored a simple integer count in the session flash. -108 lines in one commit.

The success message now correctly showed "0 messages imported. 31 duplicates skipped." The user could see the count but not the details.

Round 6: Rebuild With Cache + AJAX

The user wanted the details back. Fair enough — knowing which records were skipped is useful.

The solution: don't put the data in the session at all. Store it in the cache.

$cacheKey = 'import_duplicates_' . auth()->id() . '_' . now()->timestamp;
Cache::put($cacheKey, $result['duplicates'], now()->addMinutes(10));
session()->flash('import_duplicates_key', $cacheKey);

Session gets a lightweight string key. The actual data lives in the cache table (database-backed, 10-minute TTL). The "View Skipped" button triggers a fetch() call to a new endpoint that reads from cache and returns JSON. The modal renders client-side.

One gotcha: I initially put the new AJAX route after a wildcard route (/records/{id}/detail). Laravel tried to resolve import-duplicates as a record ID. Had to move the static route above the wildcard.

This finally worked. The session stays tiny, nginx is happy, and the user gets a full scrollable list of every skipped duplicate on demand.

What Worked

The final architecture for the import feedback:

Import service returns duplicateCount (int) and duplicates (array) separately
Controller stores the array in cache with a unique key, flashes only the key + count to session
View renders a button in the success alert that calls openDuplicatesModal()
JS fetches /records/import-duplicates?key=... and builds the table client-side
Cache auto-expires after 10 minutes — no cleanup needed

The "Applied Filters" skip was a one-liner:

$firstCellValue = strtolower(trim((string) ($rawRow[0] ?? '')));
if (str_contains($firstCellValue, 'applied filter')) {
    continue;
}

The CSV import for reference data (transaction types) was straightforward — fgetcsv loop with upsert logic. Nothing fancy, but it means the user can maintain the reference data themselves without needing a developer.

What I Learned

Never flash large arrays into session data. Even with a database session driver, the response headers can still overflow nginx's buffers. The session cookie, CSRF token, and other headers add up. If you need to pass large data between requests, use cache with a key reference.
upstream sent too big header is nginx rejecting the response from PHP-FPM. It's not a PHP error — PHP generated the response fine. Nginx's proxy_buffer_size (default 4k/8k) wasn't large enough for the response headers. You can increase it, but the better fix is to not put that much data in headers.
Static route segments must come before wildcards in Laravel. /records/import-duplicates must be registered before /records/{id}/detail, or Laravel will try to find a record with ID "import-duplicates".
Don't hardcode reference data that already exists in the database. I built a 113-entry static array when a simple pluck() would have worked. The static data was also wrong — it came from a different version of the reference document than what was actually in production.
Derive error counts from actual data, not arithmetic. I was calculating error count as skipped - duplicates, which broke when silent skips (like the "Applied Filters" row) didn't fit neatly into either category. Using count($errors) directly is more reliable.
Cache with short TTL is a good pattern for ephemeral UI data. The duplicates list only matters for a few minutes after import. Cache handles the storage and cleanup automatically. The session just holds a pointer.

Tech Notes

Stack: Laravel 12, Blade views, MySQL, nginx on managed cloud hosting
Session driver: database (but nginx buffer limits still apply to response headers)
Cache driver: database (with 10-minute TTL for import feedback)
No new dependencies — everything used built-in Laravel features
6 commits, net +262 lines across 5 files
Key pattern: session flash for lightweight signals, cache for bulk data, AJAX for on-demand loading