A 2022 Mac, 9,000 Documents, and 73 Hours: Why AI Doesn't Always Need Cutting-Edge Hardware
By Solaeris · 9 min readHow we used a first-generation Apple Silicon workstation and an open-weight AI model to build a private, local document-processing pipeline for enterprise RAG.
At Solaeris AI, we recently processed 9,000 client documents through a locally hosted large language model to extract structured metadata for a Retrieval-Augmented Generation (RAG) engine.
We used a 2022 Apple Mac Studio with an M1 Max processor and 64GB of unified memory. The job required approximately 73 hours of local AI inference, with client documents processed entirely within our isolated computing environment, without sending their contents to an external AI service.
That processing time suited our batch workload, allowing us to complete the project with the workstation we already owned.
The Unexpected AI Capabilities of Apple's M1 Max
When Apple introduced the M1 Max in 2021, the modern generative AI revolution had barely begun. ChatGPT had not yet launched, and running a 35-billion-parameter open-weight language model on a desktop computer was hardly a mainstream consideration.
Yet Apple made an architectural decision that would prove remarkably useful for local AI inference: high-bandwidth unified memory.
Unlike traditional computing systems, where the CPU and discrete GPU typically maintain separate memory pools, Apple Silicon allows its processors and GPU to access a shared pool of memory located alongside the system-on-chip.
The M1 Max provides up to 400 gigabytes per second (GB/s) of memory bandwidth.
This is significant because large language model inference frequently involves moving enormous quantities of model weights and intermediate data through memory. For memory-bound workloads, bandwidth can be a major determinant of inference performance.
The M1 Max was designed for professional graphics, video editing, and computational workloads. But its architecture also made it remarkably capable of running large AI models years after its introduction.
And with 64GB of unified memory, our workstation had sufficient capacity to load the quantized model locally while retaining room for execution overhead and working memory.
Running Qwen3.6 Locally
For this project, we selected Qwen3.6-35B-A3B-NVFP4, an open-weight Mixture-of-Experts (MoE) language model.
The model contains approximately 35 billion parameters, but its MoE architecture activates only around 3 billion parameters for each token. This reduces the computational work required during inference without requiring the full parameter set to be evaluated for every generated token.
We used the NVFP4-quantized variant through Ollama's Apple Silicon inference stack, which leverages Apple's MLX framework and Metal acceleration.
The compact model weighs approximately 24GB, making it practical to run entirely within our workstation's 64GB unified memory.
We needed to systematically analyze a large collection of client documents, identify relevant information, and produce consistent structured metadata that our RAG retrieval engine could use to locate relevant source material.
This is an ideal example of a practical enterprise AI workload: repetitive, structured, and sensitive to both processing cost and data confidentiality.
The Results: 9,000 Documents Processed
The extraction pipeline produced the following measured results:
| Performance metric | Result |
|---|---|
| Documents processed | 9,000 |
| Input token processing (prefill) | 23 tokens/sec |
| Average prefill duration | 11.2 seconds |
| Output token generation | 17 tokens/sec |
| Average extraction time per document | 29.2 seconds |
| Total processing time | Approximately 73 hours |
These figures represent our observed workload rather than a standardized hardware benchmark. Actual throughput varies with document length, output requirements, model configuration, and inference settings.
At an average of 29.2 seconds per document, processing all 9,000 documents required approximately 73 hours of aggregate inference time.
The pipeline met our application's processing requirements using hardware we already owned, with document handling and inference kept under our control.
Does a Larger Model Produce Better Results?
We also evaluated three different language models, including a dense 27-billion-parameter model, to determine whether greater computational complexity would produce meaningfully better extraction results.
For our document set and extraction requirements, we found no meaningful improvement in output quality from the more computationally demanding alternatives.
That observation does not mean smaller or MoE models always perform as well as larger dense models. Their relative strengths depend heavily on the task, data, prompting, and evaluation criteria.
For this extraction task, the evaluation gave us little reason to accept the additional processing cost of the more demanding models. We could meet the application's accuracy and reliability requirements with the model already running on our workstation.
Why Local AI Matters for Client Confidentiality
Our infrastructure decisions also had to account for client confidentiality. Client documents may contain sensitive commercial information, proprietary business processes, financial details, and other confidential material.
For this project, inference took place locally in an environment isolated from external networks.
This provided important privacy advantages:
- Client document contents did not need to be transmitted to a third-party inference API.
- The inference process did not use client data to train or fine-tune a model.
- Model execution and output storage remained under our control.
- Processing could continue without internet connectivity or dependence on a hosted AI provider.
Local inference alone is not a complete security strategy. Appropriate access controls, secure storage, audit procedures, and network isolation remain essential.
Nevertheless, running open-weight models locally offers organizations meaningful control over how sensitive information is processed.
For certain legal, financial, and enterprise workloads, that control may be as important as raw inference speed.
Memory Bandwidth: Why Old Hardware Can Still Compete
To understand why our older Mac Studio remains useful, consider the published unified-memory bandwidth of five computing platforms.
Apple's M3 Ultra provides up to 819 GB/s of unified memory bandwidth, while the M1 Max delivers 400 GB/s.
By comparison, NVIDIA's DGX Spark and AMD's Ryzen AI Halo with the Ryzen AI Max+ PRO 495 processor each deliver 273 GB/s, while the base Apple M5 provides 153 GB/s.
On memory bandwidth alone, the M1 Max offers approximately 47% more bandwidth than either DGX Spark or this Ryzen AI Halo model, while the M3 Ultra offers three times as much.
These comparisons require an important qualification: memory bandwidth is not equivalent to overall AI performance.
DGX Spark offers NVIDIA's Blackwell architecture, dedicated tensor acceleration, and a CUDA-based software ecosystem. Similarly, newer Apple Silicon chips incorporate substantial advances in compute performance and AI acceleration.
The DGX Spark may substantially outperform older Macs on workloads that benefit from its specialized hardware, despite having lower memory bandwidth. And higher-end M5-family processors differ significantly from the base M5.
Nevertheless, the comparison demonstrates why a first-generation M1 Max remains an interesting option for certain local inference workloads, even alongside newer hardware.
In particular, a well-priced used M1 Ultra with sufficient unified memory may represent an attractive alternative to purchasing new hardware when an application is constrained primarily by model memory requirements and bandwidth.
What Could That Mean for a Repetitive Task?
Consider generating a short, structured record for each of 9,000 items, one item at a time. Single-sequence token generation with a dense language model can be limited by repeatedly reading model weights from memory. NVIDIA describes why this decode phase can be memory-bound.
For an illustrative calculation, assume 200 output tokens per item and a hypothetical dense model that reads 24 GB of weights per generated token. The same model, precision and output length are used on both machines, with one active sequence, no speculative decoding, and weights already resident in memory. This is a separate scenario from the Qwen MoE extraction workload measured above.
The calculation is time = items × output tokens × GB read per token ÷ bandwidth. At 100% of published peak bandwidth, each 200-token record requires 12.0 seconds of weight reads on M1 Max and approximately 17.6 seconds on DGX Spark. Repeating the same work scales the generation time linearly:
| Records generated | M1 Max | DGX Spark |
|---|---|---|
| 1 | 12.0 sec | 17.6 sec |
| 1,000 | 3.3 hours | 4.9 hours |
| 9,000 | 30.0 hours | 44.0 hours |
Under these assumptions, M1 Max has a 31.8% shorter memory-only generation time, a difference of approximately 14 hours across 9,000 records. This is the time implication of its higher bandwidth, not a measured advantage for the complete application.
These are idealized lower bounds within this model. At 50% of peak bandwidth on both systems, the projected times double to 60.0 and 87.9 hours. Real devices may sustain different fractions of their peak bandwidth. Prefill, attention and KV-cache traffic, compute, quantization kernels, scheduling and I/O are excluded. Batching or speculative decoding also changes the weight traffic per output token, so actual performance and even the ranking can differ. The article's measured 73-hour run cannot be scaled directly by this bandwidth ratio.
Choosing Infrastructure for the Workload
AI infrastructure decisions should begin with the application.
What level of extraction accuracy is required? How large are the models? Does processing need to happen in real time, or can it run as a batch operation? How sensitive is the underlying data? And what is the actual cost of completing the workload?
A real-time application serving many users concurrently may justify dedicated GPU servers or cloud infrastructure. A batch extraction pipeline can allow more time for processing, which makes existing local hardware worth evaluating.
For our 9,000-document project, the M1 Max workstation had enough memory to run the selected model, met our extraction requirements, and kept client documents within our isolated environment. At Solaeris AI, we use these workload measurements to assess whether existing equipment can support a project and where additional infrastructure spending would help.
