this is my second progress update for the Decentralize AI Hackathon by HackerNoon and Nosana.
In my first article, I Built a Local AI Linux Assistant That Doesn't Rely on the Cloud, I introduced zkzkAgent: an AI agent that manages a Linux machine through plain language and runs entirely on my own hardware with Ollama.
That is still how it works by default. But running everything locally has a ceiling, and I kept hitting it.
qwen3.5:2b). It fits in memory and answers quickly. A 27B model does not fit. Until now, anyone on modest hardware had two options: accept a small model, or send everything to a centralized AI provider.yes.
zkzkAgent LangGraph architecture
remove_file. Only my machine can run it, and only after I type yes.models/LLM.py, and it is controlled by one setting in `.env`:MODEL_TYPE="NOSANA"
NOSANA_API_KEY="nos_..."
NOSANA_BASE_URL="https://inference.nosana.com/v1"
NOSANA_MODEL="qwen/qwen3.8-27b"
if MODEL_TYPE == "NOSANA":
API_KEY = _env("NOSANA_API_KEY")
if not API_KEY.startswith("nos_"):
raise RuntimeError("NOSANA_API_KEY missing or invalid (must start with 'nos_').")
BASE_URL = _env("NOSANA_BASE_URL", NOSANA_URL).rstrip("/")
MODEL_NAME = _env("NOSANA_MODEL") or _pick_nosana_model(BASE_URL, API_KEY)
llm = ChatOpenAI(
model=MODEL_NAME,
base_url=BASE_URL,
api_key=API_KEY,
timeout=120,
max_retries=2,
stream_usage=True, # final chunk carries token counts
)
elif MODEL_TYPE == "OLLAMA":
pass #the other implement of the code
Three details are worth explaining:
GET /models while a GPU is actually serving it. If `NOSANA_MODEL` is empty, the agent fetches that list and uses the first available model, so it never starts against a model that nobody is serving.stream_usage=True, the last streamed chunk includes token usage. That is what usage is billed on, and it is the basis for the cost numbers I plan to publish next.nosana-ollama.json, a Nosana job definition that starts the official Ollama container on a GPU node and pulls qwen3:14b. It is for people who want their own model server on Nosana instead of the shared API.To test the integration in real work, I switched to the 27B model on Nosana and gave zkzkAgent a task: create a new project called zkzk_ai, a calculator web app with a FastAPI backend and an HTML, CSS, and JavaScript frontend, and open it in VS Code.
Step 1: Route and Plan
The classifier sent the request to the planner (`NEEDS_PLANNING`). The planner wrote a 9-step plan, including the full content of every file, and then stopped and waited for me:
Reply 'ok' (or 'yes') to run this plan, or tell me what to change.
Enter your request: ok
INFO:core.agent:[ENTRY ROUTER] Pending plan approved → routing directly to execute
Step 2: Execute
Once I approved the plan, the executor got to work. It made 10 tool calls, one at a time, across 10 graph iterations (with a maximum of 15).
Here's what it actually did:
|
# |
Tool |
Action |
|---|---|---|
|
1 |
|
Created |
|
2 |
|
Wrote |
|
3 |
|
Created the |
|
4 |
|
Wrote |
|
5 |
|
Wrote |
|
6 |
|
Wrote |
|
7 |
|
Wrote |
|
8 |
|
Wrote |
|
9 |
|
Wrote |
|
10 |
|
Opened the project in VS Code |
Nothing complicated happens behind the scenes. The model running on Nosana picks a tool; that tool executes on my machine, and the result is sent back to the model.
To make this clearer, here's an actual example from the execution log (trimmed for readability):
Each call follows the same loop. The model on Nosana picks the next tool, the tool runs on my machine, and the result goes back into the conversation. In the log, one turn of that loop looks like this (trimmed):
INFO:httpx:HTTP Request: POST https://inference.nosana.com/v1/chat/completions "HTTP/1.1 200 OK"
Tools in last message: [{'name': 'create_project_folder', 'args': {'working_directory': '/home/aloha-zkaria', 'project_name': 'zkzk_ai'}}]
iteration: 0/15
INFO: tools_module.files_tools.createProjectFolder.
TOOL: Successfully created project folder /home/aloha-zkaria/zkzk_ai
Step 3: A Follow-Up Edit
Next, I asked: "Can you change the zkzk_ai Calculator to be zkzk?" The classifier sent this to the planner as well, which wrote a 7-step plan. After I approved it, the agent read static/index.html and then changed the name in three places: the page title and heading, the FastAPI app title, and the README. It did not touch the calculator logic.
Step 4: Does It Actually Run?
I started the app with `uvicorn app:app`. The page loaded, and the calculator API answered:
INFO: 127.0.0.1:52166 - "GET / HTTP/1.1" 200 OK
INFO: 127.0.0.1:52166 - "GET /static/style.css HTTP/1.1" 200 OK
INFO: 127.0.0.1:52172 - "GET /static/script.js HTTP/1.1" 200 OK
INFO: 127.0.0.1:58438 - "POST /api/calculate HTTP/1.1" 200 OK
this is the calculator app zkzkAgent built on Nosana
Across both requests, every model call in the saved log, 18 in total, went to `inference.nosana.com` and returned `200 OK`. Every file write, command, and app launch happened locally.
I reviewed the code the agent wrote, and tested the backend directly:
- No eval. Expressions are parsed with Python's ast module, and only numbers, parentheses, and `+ - * / // % **` are allowed. When I sent __import__("os"), it was rejected with "Unsupported expression element: Call".
- Clear errors. Division by zero and complex results (such as (-1)**0.5) return a readable message instead of crashing.
- Input limits. Expressions are capped at 500 characters in both the browser and the API.
- Keyboard support. Enter evaluates, Backspace deletes, Esc clears and `^` types a power.
The code is on GitHub at [zkzkGamal/calculator], and the full run log is in the [zkzkAgent repo]
My first article was about an agent that doesn't rely on the cloud. This update doesn't change that. It adds a choice. Private or sensitive work stays on local Ollama, where nothing leaves the machine. Heavy planning can use a much larger model on Nosana's network instead of a centralized provider. In both modes, the tools, the files, and the final yes stay with me.
1. A real benchmark. Run the same five tasks on the local 2B model and on the 27B model on Nosana, and measure success rate, tool calls, time, tokens, and cost. Numbers instead of impressions.
2. Hybrid routing. Because the graph has separate nodes, each node can use its own model. The classifier and chat replies are short and could stay on a small local model, while the planner and executor use Nosana. Plain conversations would then never leave the machine.
3. Automatic fallback. If Nosana can't be reached, switch to local Ollama automatically, the same way the agent already turns Wi-Fi back on by itself.
4. Close the safety gaps. Enforce plan-level confirmations. Also put `run_command` behind a confirmation or an allow-list: today it can run any shell command, and with a remote model choosing the arguments, that matters more than before.
5. Fix the calculator bug with the agent doing the fix.
git clone https://github.com/zkzkGamal/zkzkAgent.git
cd zkzkAgent
pip install -r requirements.txt
cp .env.example .env # set MODEL_TYPE="NOSANA" and your NOSANA_API_KEY
./cli.sh
Full links:
zkzkAgent: https://github.com/zkzkGamal/zkzkAgent The calculator it built: https://github.com/zkzkGamal/calculator
Full Nosana run log: https://github.com/zkzkGamal/zkzkAgent/blob/main/docs/nosana-run-2026-10-25.log
First article: https://hackernoon.com/i-built-a-local-ai-linux-assistant-that-doesnt-rely-on-the-cloud