NoteTalker for Mac now runs Porites-7B, a language model we trained for NoteTalker. It runs on your Mac, like every other part of the app.
On NoteTalker's own tasks (finding a note, adding an event, answering from your calendar, holding a conversation about your week), Porites-7B scores 61.2% overall. The general-purpose Qwen3-8B it started from scores 43.3% on the same tests. Porites-7B also starts its first reply in 10.7 seconds on a cold start, where Qwen3-8B takes 36.9, and it uses 4.46 GB of memory instead of 5.39.
(Qwen3-8B: 43.3%)
(Qwen3-8B: 36.9 s)
(Qwen3-8B: 9.2)
(Qwen3-8B: 5.39 GB)
Not just a model
Porites-7B is the result of designing three things together: the model, the prompt it reads, and the harness that runs it. Each was changed with the other two in mind, and the gains come from the combination.
The harness. NoteTalker gives the model 42 actions, such as reading a note, updating a section, searching your vault or checking your calendar. The 14 most used are always in view, and the rest load on demand. Around the model, the harness does the work that does not need a model at all. A plain "what's on today?" goes straight to the right note, and the model only phrases the answer. The harness tolerates a name you misspell by a letter when it reads a note, though never when it writes one. After a failed save, it watches for a reply that claims the save worked and replaces it with an honest one. And it keeps the prompt cached between turns, so a second reply starts in about three seconds.
The prompt. A general-purpose model needs a long system prompt full of rules: when to search, how to format a date, never to read a whole note back. NoteTalker's full prompt is 4,631 tokens, and the model has to read all of it before it can say anything. We moved those rules into the model's training instead. Porites-7B runs on a compact prompt of 1,838 tokens with the same tools and the same arguments. That shorter prompt is most of why its first reply arrives three times sooner. A general-purpose model given the compact prompt would lose the rules it depends on, so only a trained model can use it.
The model. We started from Qwen3-8B and trained it on about 3,700 worked examples of NoteTalker tasks, recorded in the app's own harness. The examples cover reading and changing notes, calendars, people, memory, multi-step conversations, plain chat, and the times when the right answer is to ask a question or say a fact isn't in your notes. We then removed a fifth of the inner width of each layer's feed-forward block and trained again so the model recovered. That leaves 7.1 billion parameters where Qwen3-8B has 8.2 billion, hence the 7B in the name. The smaller model generates faster and takes a 4.0 GB download instead of 4.6 GB. One training run takes about three and a half hours on a single RTX 3090 graphics card.
How we measured it
We ran Porites-7B and five general-purpose models through the same tests, in the same app build, on the same Mac (Apple M3, 24 GB unified memory, macOS 26.4.1). Every model ran the way NoteTalker runs it: the app's prompt and settings, the same vault of example notes, native tool calls, and "thinking" turned off. The tests are:
- Training-style tasks (144 cases): single requests like the ones the model was trained on.
- Held-out tasks (91 cases): requests the model never saw in training. These show whether it learned the job rather than the examples.
- Open-ended requests (72 cases): vague or planning questions modelled on our own hands-on testing, such as "which days are open next week?" or "do I have notes about Mary?"
- Conversations (96 checks across 10 scripted conversations of 4 to 10 turns): did each turn answer correctly, and did each note end up with the right content?
- General chat (30 cases): questions that need no tools at all, where the right behaviour is to answer and leave your notes alone.
Overall accuracy is the average of the five pass rates, so the large training set does not outweigh the small chat set. Every model ran at temperature 0, and repeat runs gave identical answers. Speed figures are the median of five rounds per model, run one after another in rotating order on a quiet machine. The comparison between models uses the build from before the 2.6.0 release so that every model ran the same code, and the 2.6.0 release build, re-measured on its own, scores 62.0% overall (95% interval 57.5 to 66.3) with a cold first reply in 11.0 seconds.
These tests measure how well a model does NoteTalker's job. They are not a general ranking of the models. Porites-7B was built for exactly these tasks, and the results show how much that is worth.
Accuracy against speed
Each dot is one model. Up is more accurate; the vertical whisker is the 95% confidence interval. NoteTalker's models are blue and the general-purpose models are gray. A hollow dot ran a reduced set of cases. Hover any dot for its numbers.
Accuracy against size
What training changed
The cleanest comparison is with Qwen3-8B, the model Porites-7B started from. Both use the same architecture, so the accuracy gap below comes from training, the compact prompt and the slimming together.
To check that a gap is real rather than luck, we compare the two models case by case and count the cases where only one of them passes. Across all 433 cases, Porites-7B passes 104 that Qwen3-8B fails, and Qwen3-8B passes 37 that Porites-7B fails. If the two models were equally good, a split that lopsided would happen by chance far less than once in a thousand tries. Porites-7B is ahead on all five test sets, by a margin that clears that bar on four of them: training-style tasks, held-out tasks, open-ended requests and general chat. On conversations it passes 69 checks to Qwen3-8B's 63, which is too close to call.
Most of the difference is in judgement rather than wording. The general-purpose model writes answers about as well, but it often answers from memory instead of reading the note, reaches for tools when you only wanted to chat, and in conversations sometimes says it saved something it never saved.
How the other models compare
Qwen3-4B is half the size and the fastest generator we tested. On NoteTalker's tasks it scores 46%, and it falls well behind on the held-out tasks it had never seen (28 of 91 against Porites-7B's 44), the best test of whether a model will cope with your requests rather than ours.
Qwen3-14B is the largest model we tested and the slowest: 75 seconds to a first reply and 8.48 GB of memory. In multi-turn conversations it does slightly better than Porites-7B (72 checks to 69, too close to call). On single requests it rarely searches your notes and sometimes calls actions that do not exist. Size alone does not teach a model how NoteTalker works.
Qwen3.5-9B is the newest model in the set and chats well, but in NoteTalker it almost never calls a tool on its own. NoteTalker's engine also cannot yet reuse its prompt cache between turns, so every reply takes about 44 seconds to start. That is a limit of our engine, not of the model. We ran it on a reduced set of cases, marked with an asterisk.
Ternary Bonsai 8B stores its weights in 2 bits, so it needs about as little memory as Qwen3-4B. The compression costs it most of its ability to choose and fill in actions, and it scores lowest of the group.
The full results
| Model | Overall (95% CI) | Cold first reply | Warm reply | Tokens/s | Peak memory | Size |
|---|---|---|---|---|---|---|
| Porites-7B | 61.2% (56.7–65.6) | 10.7 s | 3.0 s | 11.6 | 4.46 GB | 4 GB |
| Tuned Qwen3 8B (2.5.0) | 60.5% (56.4–64.6) | 34.1 s | 3.4 s | 9.1 | 5.39 GB | 4.61 GB |
| Qwen3-8B | 43.3% (38.2–48.5) | 36.9 s | 3.7 s | 9.2 | 5.39 GB | 4.61 GB |
| Qwen3-4B | 46% (41.1–50.9) | 18.5 s | 2.7 s | 14.6 | 3.26 GB | 2.26 GB |
| Qwen3-14B | 49.4% (45.5–53.3) | 74.5 s | 6.1 s | 6.2 | 8.48 GB | 7.85 GB |
| Qwen3.5-9B* | 45.5% (40.6–50.6) | 37.2 s | 43.9 s | 9.7 | 5.65 GB | 5.95 GB |
| Ternary Bonsai 8B | 39.4% (35.5–43.5) | 28.8 s | 3.2 s | 10.6 | 3.27 GB | 2.3 GB |
| Model | Training-style | Held-out | Open-ended | Conversations | Chat | Cases only Porites-7B passes / only this model passes |
|---|---|---|---|---|---|---|
| Porites-7B | 74/144 | 44/91 | 32/72 | 69/96 | 27/30 | — |
| Tuned Qwen3 8B (2.5.0) | 74/144 | 43/91 | 22/72 | 77/96 | 28/30 | 45 / 43 (p 0.915) |
| Qwen3-8B | 49/144 | 32/91 | 18/72 | 63/96 | 17/30 | 104 / 37 (p < 0.001) |
| Qwen3-4B | 60/144 | 28/91 | 15/72 | 64/96 | 21/30 | 91 / 33 (p < 0.001) |
| Qwen3-14B | 42/144 | 26/91 | 15/72 | 72/96 | 28/30 | 92 / 29 (p < 0.001) |
| Qwen3.5-9B* | 3/72 | not run | 7/48 | 28/40 | 28/30 | 58 / 11 (p < 0.001) |
| Ternary Bonsai 8B | 28/144 | 19/91 | 7/72 | 58/96 | 26/30 | 125 / 17 (p < 0.001) |
* Qwen3.5-9B ran 72 of the 144 training-style cases, 48 of the open-ended cases, 3 of the 10 conversations and all 30 chat cases; its overall figure and its comparison use only those cases. "Tuned Qwen3 8B (2.5.0)" is the model NoteTalker 2.5.0 shipped with: an earlier training round, on the full prompt and at full width. It is as accurate as Porites-7B overall (the case-by-case split is 45 to 43, which is chance), so the compact prompt and the slimming bought the speed and memory without giving up accuracy. The p-value is an exact two-sided sign test on the cases where only one of the two models passes.
Known limits
- Mac only. Porites-7B runs on Apple Silicon Macs. NoteTalker on Windows and Linux keeps its current models.
- Short answers. Asked for about 300 words, Porites-7B wrote 82, where the general-purpose models wrote 206 to 398. Training for NoteTalker's mostly short answers made it terse, and longer writing is something we are working on.
- NoteTalker's tasks, not every task. These results say which model does NoteTalker's job best. For other work, a general-purpose model may suit you better.
You choose the model in Settings, and the larger general-purpose models remain optional downloads if you want to compare them on your own notes.