agentlang-index · task medium
POST a JSON pair and extract the sum
013-http-json-sum. Read three newline-terminated lines from standard input:
Prompt
This is the natural-language brief given to every model, verbatim. The harness prefixes a language-specific calling-convention block and suffixes a "return only the source code" instruction. Nothing else.
# 013-http-json-sum
Read three newline-terminated lines from standard input:
1. A URL — the endpoint to POST to.
2. An integer `a`.
3. An integer `b`.
Issue an HTTP POST request to the URL with:
- A 5000 millisecond transport timeout.
- Header `Content-Type: application/json`.
- Body `{"a":<a>,"b":<b>}` (no whitespace inside the JSON).
On a transport-successful HTTP 200 response whose body parses as a
JSON object containing an integer `sum` field, write that integer
as decimal followed by a newline to standard output (e.g. `7\n`).
On any failure — transport error (DNS, connect, TLS, timeout,
invalid URL, unsupported protocol, provider unavailable, I/O),
non-200 status, JSON parse error, or `sum` field missing /
non-integer — write the literal string `error\n` to standard
output instead. Do not write to standard error. Exit with status 0
in every case.
All integers (a, b, sum) fit in `i32`. Trailing whitespace on the
two digit lines should be trimmed before use.
## Examples
Input (stdin, three lines):
```
http://localhost:18013/sum
3
4
```
Output (stdout):
```
7
```
Input (stdin):
```
http://localhost:18013/sum
100
200
```
Output (stdout):
```
300
```
Input (stdin):
```
http://localhost:18013/404
1
1
```
Output (stdout):
```
error
```
Input (stdin):
```
http://this-host-does-not-resolve.invalid/sum
1
1
```
Output (stdout):
```
error
```
## Acceptance
- stdout matches the expected bytes exactly per test case
- stderr is empty
- exit code is 0
- the run completes within 10 seconds wall time
## Input convention by language
- **TypeScript / Rust / Go / Python**: read three lines from stdin.
- **Zero**: take URL from `argv[1]`, a from `argv[2]`, b from
`argv[3]` because Zero 0.1.2 does not expose a standard-input
capability. The byte semantics are the same — Zero callers
should treat the three argv strings identically to three stdin
reads.
## Verifier fixture
The verifier starts a local Python HTTP fixture server on port
18013 before running the references and tears it down after.
- `POST /sum` — parses JSON body `{"a":n,"b":m}`, returns
`{"sum":n+m}` with HTTP 200 and `Content-Type: application/json`.
- `POST /missing` — returns `{"other":99}` (no `sum` field) with
HTTP 200.
- `POST /badjson` — returns the literal body `not-json` with
HTTP 200.
- `POST /404` — returns HTTP 404 with an empty body.
References target `http://127.0.0.1:18013/<path>` for the
happy-path and structured-failure cases. The transport-failure
case points at a name that does not resolve.
Acceptance
A task counts as passed only when every public and hidden test case agrees on these fields. No fuzzy matching, no "off by one trailing newline is fine."
| stdout (byte-exact, per case) | true |
|---|---|
| stderr (exact bytes) | "" |
| exit code | 0 |
| wall time max (ms) | 10000 |
| tags | networking, http, json, stdlib-breadth |
Results
Each cell is one attempt. Pass means stdout matched byte-exact on every test case, stderr empty, exit zero. Hover a failure to see the captured first line of the diagnostic.
| Model | Zero | TypeScript | Rust | Go | Python |
|---|---|---|---|---|---|
| gpt-4o | compile | ✓ | wrong output | ✓ | other |
| gpt-4o-mini | compile | wrong output | wrong output | ✓ | other |
| gpt-5 | wrong output | ✓ | wrong output | ✓ | ✓ |
Failure excerpts
9 of 15 attempts failed. Each card is one attempt, with the captured first line of the diagnostic.
-
ref.zero:1:1 IMP001: unknown package-local import 'std::net' -
(no diagnostic captured) -
Traceback (most recent call last): -
ref.zero:1:1 IMP001: unknown package-local import 'lib http' -
(no diagnostic captured) -
(no diagnostic captured) -
Traceback (most recent call last): -
(no diagnostic captured) -
(no diagnostic captured)
Reference implementations
The hand-written reference each language ships with. Every reference passes the same public and hidden test suite under the pinned toolchain before any model touches the task.
Click a language to expand
Zero 574 lines
// HTTP POST + JSON sum reference for AgentLang Index.
//
// Zero 0.1.2 has no exposed stdin, so URL/a/b come from argv[1..3].
// std.http.fetch is invoked with a POST envelope (method-line + JSON
// content-type header + JSON body). On HTTP 200 we scan the response
// body for `"sum":<digits>` and write that integer + newline. On any
// transport, status, or parse failure we write `error\n`.
//
// The direct backend MVP forbids user functions taking Span<u8>,
// MutSpan<u8>, or returning shape locals, so the i32 parse, envelope
// rendering, response scan, and output rendering are all inline here.
pub fun main(world: World) -> Void raises {
let maybe_url = std.args.get(1)
let maybe_a = std.args.get(2)
let maybe_b = std.args.get(3)
if maybe_url.has == false {
check world.out.write("error\n")
return
}
if maybe_a.has == false {
check world.out.write("error\n")
return
}
if maybe_b.has == false {
check world.out.write("error\n")
return
}
let url_s = std.mem.span(maybe_url.value)
let a_s = std.mem.span(maybe_a.value)
let b_s = std.mem.span(maybe_b.value)
// ---- Parse a as i32 from trimmed ASCII digits ----
let a_n = std.mem.len(a_s)
let mut a_i: usize = 0
let mut a_skip_flag: Bool = true
while a_skip_flag {
if a_i >= a_n {
a_skip_flag = false
} else {
let c = a_s[a_i]
if c == 32_u8 {
a_i = a_i + 1
} else {
if c == 9_u8 {
a_i = a_i + 1
} else {
a_skip_flag = false
}
}
}
}
if a_i >= a_n {
check world.out.write("error\n")
return
}
let mut a_neg: Bool = false
if a_s[a_i] == 45_u8 {
a_neg = true
a_i = a_i + 1
}
if a_i >= a_n {
check world.out.write("error\n")
return
}
let mut a_have_digit: Bool = false
let mut a_acc: u32 = 0_u32
let mut a_digit_flag: Bool = true
while a_digit_flag {
if a_i >= a_n {
a_digit_flag = false
} else {
let c = a_s[a_i]
if c >= 48_u8 {
if c <= 57_u8 {
let d: u32 = (c - 48_u8) as u32
a_acc = a_acc * 10_u32 + d
a_have_digit = true
a_i = a_i + 1
} else {
a_digit_flag = false
}
} else {
a_digit_flag = false
}
}
}
let mut a_tail_flag: Bool = true
let mut a_tail_ok: Bool = true
while a_tail_flag {
if a_i >= a_n {
a_tail_flag = false
} else {
let c = a_s[a_i]
if c == 32_u8 {
a_i = a_i + 1
} else {
if c == 9_u8 {
a_i = a_i + 1
} else {
if c == 10_u8 {
a_i = a_i + 1
} else {
if c == 13_u8 {
a_i = a_i + 1
} else {
a_tail_ok = false
a_tail_flag = false
}
}
}
}
}
}
if a_tail_ok == false {
check world.out.write("error\n")
return
}
if a_have_digit == false {
check world.out.write("error\n")
return
}
if a_acc > 2147483647_u32 {
check world.out.write("error\n")
return
}
let mut a_val: i32 = a_acc as i32
if a_neg {
a_val = 0_i32 - a_val
}
// ---- Parse b as i32 from trimmed ASCII digits ----
let b_n = std.mem.len(b_s)
let mut b_i: usize = 0
let mut b_skip_flag: Bool = true
while b_skip_flag {
if b_i >= b_n {
b_skip_flag = false
} else {
let c = b_s[b_i]
if c == 32_u8 {
b_i = b_i + 1
} else {
if c == 9_u8 {
b_i = b_i + 1
} else {
b_skip_flag = false
}
}
}
}
if b_i >= b_n {
check world.out.write("error\n")
return
}
let mut b_neg: Bool = false
if b_s[b_i] == 45_u8 {
b_neg = true
b_i = b_i + 1
}
if b_i >= b_n {
check world.out.write("error\n")
return
}
let mut b_have_digit: Bool = false
let mut b_acc: u32 = 0_u32
let mut b_digit_flag: Bool = true
while b_digit_flag {
if b_i >= b_n {
b_digit_flag = false
} else {
let c = b_s[b_i]
if c >= 48_u8 {
if c <= 57_u8 {
let d: u32 = (c - 48_u8) as u32
b_acc = b_acc * 10_u32 + d
b_have_digit = true
b_i = b_i + 1
} else {
b_digit_flag = false
}
} else {
b_digit_flag = false
}
}
}
let mut b_tail_flag: Bool = true
let mut b_tail_ok: Bool = true
while b_tail_flag {
if b_i >= b_n {
b_tail_flag = false
} else {
let c = b_s[b_i]
if c == 32_u8 {
b_i = b_i + 1
} else {
if c == 9_u8 {
b_i = b_i + 1
} else {
if c == 10_u8 {
b_i = b_i + 1
} else {
if c == 13_u8 {
b_i = b_i + 1
} else {
b_tail_ok = false
b_tail_flag = false
}
}
}
}
}
}
if b_tail_ok == false {
check world.out.write("error\n")
return
}
if b_have_digit == false {
check world.out.write("error\n")
return
}
if b_acc > 2147483647_u32 {
check world.out.write("error\n")
return
}
let mut b_val: i32 = b_acc as i32
if b_neg {
b_val = 0_i32 - b_val
}
// ---- Build POST envelope: POST <url>\n + Content-Type header + body ----
let mut envelope: [1024]u8 = [0_u8; 1024]
let mut env_n: usize = 0
let prefix = std.mem.span("POST ")
let pre_n = std.mem.len(prefix)
let mut pi: usize = 0
while pi < pre_n {
envelope[env_n] = prefix[pi]
env_n = env_n + 1
pi = pi + 1
}
let url_n = std.mem.len(url_s)
let mut ui: usize = 0
let mut url_flag: Bool = true
while url_flag {
if ui >= url_n {
url_flag = false
} else {
let c = url_s[ui]
if c == 10_u8 {
url_flag = false
} else {
if c == 13_u8 {
url_flag = false
} else {
envelope[env_n] = c
env_n = env_n + 1
ui = ui + 1
}
}
}
}
envelope[env_n] = 10_u8
env_n = env_n + 1
let ct = std.mem.span("Content-Type: application/json\n")
let ct_n = std.mem.len(ct)
let mut ci: usize = 0
while ci < ct_n {
envelope[env_n] = ct[ci]
env_n = env_n + 1
ci = ci + 1
}
envelope[env_n] = 10_u8
env_n = env_n + 1
// Body: {"a":<a>,"b":<b>}
envelope[env_n] = 123_u8
env_n = env_n + 1
envelope[env_n] = 34_u8
env_n = env_n + 1
envelope[env_n] = 97_u8
env_n = env_n + 1
envelope[env_n] = 34_u8
env_n = env_n + 1
envelope[env_n] = 58_u8
env_n = env_n + 1
// Inline writeI32 for a_val
let mut a_write_neg: Bool = false
let mut a_write_v: u32 = 0_u32
if a_val < 0_i32 {
envelope[env_n] = 45_u8
env_n = env_n + 1
a_write_neg = true
a_write_v = (0_i32 - a_val) as u32
} else {
a_write_v = a_val as u32
}
if a_write_v == 0_u32 {
envelope[env_n] = 48_u8
env_n = env_n + 1
} else {
let mut a_tmp: [16]u8 = [0_u8; 16]
let mut a_tn: usize = 0
while a_write_v > 0_u32 {
let digit_u32: u32 = 48_u32 + (a_write_v % 10_u32)
a_tmp[a_tn] = digit_u32 as u8
a_tn = a_tn + 1
a_write_v = a_write_v / 10_u32
}
let mut a_wj: usize = 0
while a_wj < a_tn {
envelope[env_n] = a_tmp[a_tn - 1 - a_wj]
env_n = env_n + 1
a_wj = a_wj + 1
}
}
envelope[env_n] = 44_u8
env_n = env_n + 1
envelope[env_n] = 34_u8
env_n = env_n + 1
envelope[env_n] = 98_u8
env_n = env_n + 1
envelope[env_n] = 34_u8
env_n = env_n + 1
envelope[env_n] = 58_u8
env_n = env_n + 1
// Inline writeI32 for b_val
let mut b_write_neg: Bool = false
let mut b_write_v: u32 = 0_u32
if b_val < 0_i32 {
envelope[env_n] = 45_u8
env_n = env_n + 1
b_write_neg = true
b_write_v = (0_i32 - b_val) as u32
} else {
b_write_v = b_val as u32
}
if b_write_v == 0_u32 {
envelope[env_n] = 48_u8
env_n = env_n + 1
} else {
let mut b_tmp: [16]u8 = [0_u8; 16]
let mut b_tn: usize = 0
while b_write_v > 0_u32 {
let digit_u32: u32 = 48_u32 + (b_write_v % 10_u32)
b_tmp[b_tn] = digit_u32 as u8
b_tn = b_tn + 1
b_write_v = b_write_v / 10_u32
}
let mut b_wj: usize = 0
while b_wj < b_tn {
envelope[env_n] = b_tmp[b_tn - 1 - b_wj]
env_n = env_n + 1
b_wj = b_wj + 1
}
}
envelope[env_n] = 125_u8
env_n = env_n + 1
// ---- Fetch ----
let net = std.net.host()
let client = std.http.client(net)
let mut response: [8192]u8 = [0_u8; 8192]
let result = std.http.fetch(client, envelope[0..env_n], response, std.time.ms(5000))
if std.http.resultOk(result) == false {
check world.out.write("error\n")
return
}
if std.http.resultStatus(result) != 200 {
check world.out.write("error\n")
return
}
let body_len = std.http.resultBodyLen(result)
let body_offset = std.http.responseBodyOffset(response)
// ---- Scan body for `"sum":<digits>` ----
let needle = std.mem.span("\"sum\":")
let needle_n = std.mem.len(needle)
let end: usize = body_offset + body_len
if needle_n == 0_usize {
check world.out.write("error\n")
return
}
if end < needle_n {
check world.out.write("error\n")
return
}
let stop: usize = end - needle_n
let mut found_at: usize = 0_usize
let mut found: Bool = false
let mut si: usize = body_offset
let mut search_flag: Bool = true
while search_flag {
if si > stop {
search_flag = false
} else {
let mut j: usize = 0
let mut matched: Bool = true
let mut inner_flag: Bool = true
while inner_flag {
if j >= needle_n {
inner_flag = false
} else {
if response[si + j] != needle[j] {
matched = false
inner_flag = false
} else {
j = j + 1
}
}
}
if matched {
found_at = si + needle_n
found = true
search_flag = false
} else {
si = si + 1
}
}
}
if found == false {
check world.out.write("error\n")
return
}
// Parse digits (with optional leading `-`) at found_at.
let mut k = found_at
let mut k_ws_flag: Bool = true
while k_ws_flag {
if k >= end {
k_ws_flag = false
} else {
let c = response[k]
if c == 32_u8 {
k = k + 1
} else {
if c == 9_u8 {
k = k + 1
} else {
k_ws_flag = false
}
}
}
}
if k >= end {
check world.out.write("error\n")
return
}
let mut sum_neg: Bool = false
if response[k] == 45_u8 {
sum_neg = true
k = k + 1
}
if k >= end {
check world.out.write("error\n")
return
}
let mut sum_have_digit: Bool = false
let mut sum_acc: u32 = 0_u32
let mut sum_digit_flag: Bool = true
while sum_digit_flag {
if k >= end {
sum_digit_flag = false
} else {
let c = response[k]
if c >= 48_u8 {
if c <= 57_u8 {
let d: u32 = (c - 48_u8) as u32
sum_acc = sum_acc * 10_u32 + d
sum_have_digit = true
k = k + 1
} else {
sum_digit_flag = false
}
} else {
sum_digit_flag = false
}
}
}
if sum_have_digit == false {
check world.out.write("error\n")
return
}
// After digits, next byte must be a JSON terminator or end.
let mut term_ok: Bool = true
if k < end {
let c = response[k]
if c == 44_u8 {
// ok
} else {
if c == 125_u8 {
// ok
} else {
if c == 93_u8 {
// ok
} else {
if c == 32_u8 {
// ok
} else {
if c == 9_u8 {
// ok
} else {
if c == 10_u8 {
// ok
} else {
if c == 13_u8 {
// ok
} else {
term_ok = false
}
}
}
}
}
}
}
}
if term_ok == false {
check world.out.write("error\n")
return
}
if sum_acc > 2147483647_u32 {
check world.out.write("error\n")
return
}
let mut sum_val: i32 = sum_acc as i32
if sum_neg {
sum_val = 0_i32 - sum_val
}
// ---- Render sum_val + newline ----
let mut out: [16]u8 = [0_u8; 16]
let mut out_n: usize = 0
let mut out_v: u32 = 0_u32
if sum_val < 0_i32 {
out[out_n] = 45_u8
out_n = out_n + 1
out_v = (0_i32 - sum_val) as u32
} else {
out_v = sum_val as u32
}
if out_v == 0_u32 {
out[out_n] = 48_u8
out_n = out_n + 1
} else {
let mut out_tmp: [16]u8 = [0_u8; 16]
let mut out_tn: usize = 0
while out_v > 0_u32 {
let digit_u32: u32 = 48_u32 + (out_v % 10_u32)
out_tmp[out_tn] = digit_u32 as u8
out_tn = out_tn + 1
out_v = out_v / 10_u32
}
let mut out_wj: usize = 0
while out_wj < out_tn {
out[out_n] = out_tmp[out_tn - 1 - out_wj]
out_n = out_n + 1
out_wj = out_wj + 1
}
}
out[out_n] = 10_u8
out_n = out_n + 1
check world.out.write(out[0..out_n])
return
}
TypeScript 92 lines
// HTTP POST + JSON sum reference for AgentLang Index.
//
// Reads three stdin lines: URL, a, b. POSTs `{"a":<a>,"b":<b>}` to URL
// with Content-Type: application/json and 5000 ms timeout. On HTTP 200
// with a JSON body containing an integer `sum`, writes the sum and
// newline. Otherwise writes `error\n`. Always exits 0.
async function readStdin(): Promise<string> {
const chunks: Buffer[] = [];
for await (const chunk of process.stdin as AsyncIterable<Buffer>) {
chunks.push(chunk);
}
return Buffer.concat(chunks).toString("utf-8");
}
function fail(): void {
process.stdout.write("error\n");
}
async function main(): Promise<void> {
let url: string;
let a: number;
let b: number;
try {
const lines = (await readStdin()).split(/\r?\n/);
url = (lines[0] ?? "").trim();
a = parseInt((lines[1] ?? "").trim(), 10);
b = parseInt((lines[2] ?? "").trim(), 10);
if (!url || !Number.isInteger(a) || !Number.isInteger(b)) {
fail();
return;
}
} catch {
fail();
return;
}
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), 5000);
let resp: Response;
try {
resp = await fetch(url, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: `{"a":${a},"b":${b}}`,
signal: controller.signal,
redirect: "manual",
});
} catch {
clearTimeout(timeout);
fail();
return;
}
clearTimeout(timeout);
if (resp.status !== 200) {
fail();
return;
}
let bodyText: string;
try {
bodyText = await resp.text();
} catch {
fail();
return;
}
let parsed: unknown;
try {
parsed = JSON.parse(bodyText);
} catch {
fail();
return;
}
if (
typeof parsed !== "object" ||
parsed === null ||
!("sum" in parsed) ||
typeof (parsed as { sum: unknown }).sum !== "number" ||
!Number.isInteger((parsed as { sum: number }).sum)
) {
fail();
return;
}
const s = (parsed as { sum: number }).sum;
process.stdout.write(`${s}\n`);
}
main();
Rust 50 lines
// HTTP POST + JSON sum reference for AgentLang Index.
use std::io::{self, Read};
use std::time::Duration;
fn fail() {
print!("error\n");
}
fn run() -> Option<i64> {
let mut input = String::new();
io::stdin().read_to_string(&mut input).ok()?;
let mut lines = input.lines();
let url = lines.next()?.trim().to_string();
let a: i64 = lines.next()?.trim().parse().ok()?;
let b: i64 = lines.next()?.trim().parse().ok()?;
let body = format!("{{\"a\":{},\"b\":{}}}", a, b);
let agent = ureq::Agent::config_builder()
.timeout_global(Some(Duration::from_secs(5)))
.build()
.new_agent();
let response = agent
.post(&url)
.header("Content-Type", "application/json")
.send(body.as_bytes());
let mut response = match response {
Ok(r) => r,
Err(_) => return None,
};
if response.status().as_u16() != 200 {
return None;
}
let body_text = response.body_mut().read_to_string().ok()?;
let parsed: serde_json::Value = serde_json::from_str(&body_text).ok()?;
let obj = parsed.as_object()?;
let sum_val = obj.get("sum")?;
let s = sum_val.as_i64()?;
Some(s)
}
fn main() {
match run() {
Some(s) => print!("{}\n", s),
None => fail(),
}
}
Go 95 lines
// HTTP POST + JSON sum reference for AgentLang Index.
package main
import (
"bufio"
"bytes"
"encoding/json"
"fmt"
"net/http"
"os"
"strconv"
"strings"
"time"
)
func fail() {
fmt.Print("error\n")
}
func run() bool {
scanner := bufio.NewScanner(os.Stdin)
scanner.Buffer(make([]byte, 0, 4096), 1<<20)
var lines []string
for scanner.Scan() {
lines = append(lines, scanner.Text())
if len(lines) == 3 {
break
}
}
if len(lines) < 3 {
return false
}
url := strings.TrimSpace(lines[0])
a, err := strconv.Atoi(strings.TrimSpace(lines[1]))
if err != nil {
return false
}
b, err := strconv.Atoi(strings.TrimSpace(lines[2]))
if err != nil {
return false
}
if url == "" {
return false
}
body := []byte(fmt.Sprintf(`{"a":%d,"b":%d}`, a, b))
req, err := http.NewRequest(http.MethodPost, url, bytes.NewReader(body))
if err != nil {
return false
}
req.Header.Set("Content-Type", "application/json")
client := &http.Client{
Timeout: 5 * time.Second,
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
}
resp, err := client.Do(req)
if err != nil {
return false
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
return false
}
var parsed map[string]json.RawMessage
dec := json.NewDecoder(resp.Body)
if err := dec.Decode(&parsed); err != nil {
return false
}
rawSum, ok := parsed["sum"]
if !ok {
return false
}
var n json.Number
if err := json.Unmarshal(rawSum, &n); err != nil {
return false
}
s, err := n.Int64()
if err != nil {
return false
}
fmt.Printf("%d\n", s)
return true
}
func main() {
if !run() {
fail()
}
}
Python 54 lines
#!/usr/bin/env python3
"""HTTP POST + JSON sum reference for AgentLang Index."""
import json
import sys
import urllib.error
import urllib.request
def main() -> int:
try:
lines = sys.stdin.read().splitlines()
url = lines[0].strip()
a = int(lines[1].strip())
b = int(lines[2].strip())
except (IndexError, ValueError, OSError):
sys.stdout.write("error\n")
return 0
body = f'{{"a":{a},"b":{b}}}'.encode("ascii")
req = urllib.request.Request(
url,
data=body,
method="POST",
headers={"Content-Type": "application/json"},
)
try:
with urllib.request.urlopen(req, timeout=5) as resp:
if resp.status != 200:
sys.stdout.write("error\n")
return 0
response_body = resp.read()
except urllib.error.HTTPError:
sys.stdout.write("error\n")
return 0
except (urllib.error.URLError, ValueError, OSError, TimeoutError):
sys.stdout.write("error\n")
return 0
try:
parsed = json.loads(response_body.decode("utf-8"))
if not isinstance(parsed, dict):
raise ValueError
s = parsed.get("sum")
if not isinstance(s, int) or isinstance(s, bool):
raise ValueError
sys.stdout.write(f"{s}\n")
except (json.JSONDecodeError, ValueError, UnicodeDecodeError):
sys.stdout.write("error\n")
return 0
if __name__ == "__main__":
sys.exit(main())
Design notes
Algorithm, failure modes, cross-language parity, and where Zero needed a workaround. From corpus/013-http-json-sum/notes.md.
Algorithm
Read three inputs (URL, integer a, integer b). POST a JSON body
{"a":a,"b":b} to the URL with Content-Type: application/json. On
HTTP 200, parse the response body as JSON, extract integer field
"sum", and write that integer followed by \n. On any transport
error, non-200 status, body parse failure, or missing sum field,
write error\n. Process exit is 0 either way.
Fixture
A local Python HTTP server listens on 127.0.0.1:18013 while
verify.sh runs the references. Routes are all POST:
/sum→ parses request body as JSON, returns{"sum":a+b}200/missing→ returns{"other":99}200 (nosumfield)/badjson→ returns the literal bodynot-jsonwith status 200/404→ returns HTTP 404 with empty body
verify.sh starts the fixture in the background, waits for ready
on stdout, runs all references against six cases (three public,
three hidden) covering the four routes, and kills the fixture on
exit.
Edge cases
- Inputs are trimmed for leading/trailing whitespace before integer parsing.
aandbare parsed as i32 with optional leading-. Values outside i32 range are rejected (writeerror\n).- The
"sum":byte literal is matched in the response body; the digits that follow are parsed with optional leading-and i32 range validation. - The byte after the digits must be a JSON terminator (
,,},], whitespace, or end-of-buffer); a trailing non-terminator byte rejects the parse. - Redirects are not followed in any reference.
- Five second wall-clock timeout on every fetch.
Zero-specific notes
- argv[1..3] carry URL, a, b because Zero 0.1.2 has no exposed stdin.
- The POST envelope shape for
std.http.fetchisPOST <url>\nContent-Type: application/json\n\n{"a":<a>,"b":<b>}in a[1024]u8buffer. - Zero 0.1.2's direct backend ELF64 MVP forbids user functions that
take or return shape values,
Span<u8>, orMutSpan<u8>. The only fully supported user-function signatures are primitive integer + Bool parameters and primitive integer + Bool returns. Member access on shape values is supported only forMaybe<MutSpan<u8>>.hasand.value. Every other shape access routes throughCGEN004. The ref therefore inlines the integer parse (twice, once foraand once forb), the envelope decimal renderer (twice), the response body scan, and the output renderer all directly intomain. The same code in a helper-function shape is fine forzero checkbut fails atzero buildtime. std.jsonin Zero 0.1.2 exposesvalidate,parse, andstreamTokensbut no field accessors. The ref scans for the byte literal"sum":and parses the integer that follows, validating the following byte is a JSON terminator.- The
(c - 48_u8) as u32cast must be parenthesized AND bound to a typed local (let digit: u32 = ...) before being added to au32accumulator; the inline formacc * 10_u32 + (c - 48_u8) as u32trips TYP002 becauseasdoes not lift the resulting u8 cleanly in mixed-type expressions. 0_i32 - 2147483648_i64 as i32is rejected as type-mismatched. The ref drops INT_MIN special-casing: any |value| > 2147483647 is anerror\n, matching the i32-range contract of the other references.std.http.fetchis provided by libcurl at link time. The userland install from task 012 (~/.local/include/curl/,~/.local/lib/libcurl.sosymlink,C_INCLUDE_PATHandLIBRARY_PATHexports in~/.config/truffle/env.sh) carries over unchanged.- main ends with an explicit
returnto dodge the trailing-write byte-count-as-exit-code codegen quirk surfaced in task 012.
Cross-implementation parity
All five references issue exactly one POST with body
{"a":<a>,"b":<b>}, observe one HTTP transaction (no redirects),
and surface either the integer sum field of the response object
or the literal error\n for any failure. Byte-exact agreement on
every case.
Cost
| Model | Prompt tokens | Completion tokens | API ms |
|---|---|---|---|
| gpt-4o | 4,755 | 1,716 | 14,778 |
| gpt-4o-mini | 4,755 | 1,612 | 25,176 |
| gpt-5 | 4,750 | 25,418 | 229,604 |
Tokens and API ms are summed across the five languages this model attempted for this task.
Compare
Model deep-dives: gpt-4o · gpt-4o-mini · gpt-5 . Back to the leaderboard and methodology.