InferenceFireworks separates global capacity placement from the inference request path
Its September 28 architecture account reports an observational reliability gain, while preserving location constraints and geographic latency.
Inference
How models reason and run: serving speed, memory, inference-time compute and cost per useful result.
Serving & runtime · Compression & efficiency · Inference-time reasoning
12 articles
Inference
InferenceModal’s Endpoint Candidates make inference changes testable beside production
The October 12 release mirrors sampled requests, but best-effort drops must remain visible in the comparison.
InferenceLiquid AI’s LFM2.5 puts local agent performance under a workload-specific test
Sparse activation lowers per-token work, while total memory and completed-task cost still govern deployment.
InferenceBaseten’s agent-built inference engines make correctness part of the speed target
The reported gains concern constrained experiments; the engines were not serving production traffic in the October account.
InferenceTogether's IBM expansion puts the inference service behind the chips
Together AI's October update describes an IBM Cloud-NVIDIA collaboration; IBM's original agreement sets a Q1 2027 availability expectation, so capacity plans must remain distinct from a generally available service.
InferenceTogether's CPD gives reused context its own fast path
A March research release separates cold prompts from cache-backed requests; its reported throughput gains apply to a specified mixed workload, not every inference service.
Hover to animateInferenceLiquid AI can answer more questions without looking at the scene again
Liquid AI’s d1 measurements suggest several decisions can share the cost of reading a scene. The saving depends on how quickly that scene changes.
Hover to animateInferenceFast transcription does not always mean a faster voice response
QVAC’s new on-device recogniser makes the distinction between bulk throughput, short-utterance latency and first-load cost unusually visible.
Hover to animateInferenceThe cheapest next model may be the one that already holds your cache
Fireworks’ new router weighs the price of switching models against the cached work a session would lose. Its internal test shows substantial savings with a modest accuracy trade-off.
Hover to animateInferenceMistral makes document navigation a way to spend fewer tokens
Agentic Search gives models tools to inspect documents after finding them. Company tests suggest those extra capabilities can replace costly repeated searches.
Hover to animateInferenceDFlash 2: A small model guesses ahead so the main model can work faster
A small model proposes several tokens at once; the main model checks them. Better accepted guesses can reduce expensive model passes without handing the final answer to the smaller model.
Hover to animateInferencePrismML: A smaller model could run on hardware you already own
Bonsai 2’s reported 5.9 GB footprint raises a deployment question: can the compressed model meet your task standard locally, after context memory and fallback costs are included?
No articles in this topic yet
Explore the other topics in this desk.
Explore the research library for paper-based analysis.