All ideasMore in Work
Work

A week of the AI tuning tripled his local model's speed on old datacentre cards

The AI had been optimizing llama.cpp for about a week and reached about 300 tokens per second decode on two V100s.

How it works

  1. The AI optimizes the engine for about a week
  2. Reaches about 300 tokens per second decode on two older datacentre GPUs

Why it’s worth knowing

A week of small wins is what optimization actually looks like.

Where it’s from

Summarized from a public post on news.ycombinator.com. See the original ↗

Upgraded prompt

A ready-to-use prompt for this idea, built from research on what makes AI agents do the job right: who the AI should be, the exact steps, and a check to make sure it was done the way you wanted.

1Who the AI should be
2The prompt, with the steps
3The check when it’s done
Upgraded prompts are coming soon.

Try next

WorkFind me 48 school-district customers and draft each pitch — one replied, booked and paidWorkThe AI audited my eBay store, found 28 listings going nowhere, and sold a bundle for meWorkA contractor talks measurements into his phone and QuickBooks sends the estimate in 45 secondsWorkRead every job advert, tailor my CV and apply with my approval

Every idea is a real post from someone who tried it, summarized in our words, with a link to the original. App names are swapped for plain words.