agentlang-index · task medium
GET a URL and write the HTTP status code
012-http-status-code. Read a single URL from standard input.
Prompt
This is the natural-language brief given to every model, verbatim. The harness prefixes a language-specific calling-convention block and suffixes a "return only the source code" instruction. Nothing else.
# 012-http-status-code
Read a single URL from standard input. Issue an HTTP GET request to
that URL with a 5000 millisecond transport timeout. Write the HTTP
response status code as decimal followed by a newline to standard
output (e.g. `200\n`).
If the transport fails for any reason — DNS lookup failure, connect
refused, TLS error, request timed out, invalid URL, unsupported
protocol, the HTTP provider being unavailable, or any I/O error —
write the literal string `error\n` to standard output instead of a
status code. Do not write to standard error in either case. Exit
with status 0 in every case.
The URL on stdin may have a trailing newline; trim it before use.
## Examples
Input (stdin):
```
http://localhost:18012/status/200
```
Output (stdout):
```
200
```
Input (stdin):
```
http://localhost:18012/status/404
```
Output (stdout):
```
404
```
Input (stdin):
```
http://this-host-does-not-resolve.invalid/
```
Output (stdout):
```
error
```
## Acceptance
- stdout matches the expected bytes exactly per test case
- stderr is empty
- exit code is 0
- the run completes within 10 seconds wall time
## Input convention by language
- **TypeScript / Rust / Go / Python**: read the URL from stdin until
newline or EOF; trim trailing newline.
- **Zero**: take the URL from `argv[1]` because Zero 0.1.2 does not
expose a standard-input capability. The byte semantics are the
same — Zero callers should treat the argv string identically to a
stdin read.
## Verifier fixture
The verifier starts a local Python HTTP fixture server on port 18012
before running the references and tears it down after. The server
responds to `/status/<code>` with status `<code>` and an empty body,
to `/` with 200 OK, and to any other path with 200 OK. References
target `http://localhost:18012/...` for the happy-path cases. For
the transport-failure cases the URL points at a name that does not
resolve, so the reference must surface the failure as `error\n`.
Acceptance
A task counts as passed only when every public and hidden test case agrees on these fields. No fuzzy matching, no "off by one trailing newline is fine."
| stdout (byte-exact, per case) | true |
|---|---|
| stderr (exact bytes) | "" |
| exit code | 0 |
| wall time max (ms) | 10000 |
| tags | networking, http, stdlib-breadth |
Results
Each cell is one attempt. Pass means stdout matched byte-exact on every test case, stderr empty, exit zero. Hover a failure to see the captured first line of the diagnostic.
| Model | Zero | TypeScript | Rust | Go | Python |
|---|---|---|---|---|---|
| gpt-4o | compile | ✓ | wrong output | ✓ | other |
| gpt-4o-mini | compile | wrong output | wrong output | ✓ | other |
| gpt-5 | compile | ✓ | ✓ | ✓ | ✓ |
Failure excerpts
8 of 15 attempts failed. Each card is one attempt, with the captured first line of the diagnostic.
-
ref.zero:1:1 IMP001: unknown package-local import 'std::net' -
(no diagnostic captured) -
Traceback (most recent call last): -
ref.zero:1:1 IMP001: unknown package-local import 'lib http' -
(no diagnostic captured) -
(no diagnostic captured) -
Traceback (most recent call last): -
ref.zero:1:1 IMP001: unknown package-local import 'std'
Reference implementations
The hand-written reference each language ships with. Every reference passes the same public and hidden test suite under the pinned toolchain before any model touches the task.
Click a language to expand
Zero 90 lines
// HTTP GET status-code reference for AgentLang Index.
//
// Zero 0.1.2 has no exposed stdin, so the URL is read from argv[1].
// std.http.fetch is invoked with a `GET <url>\n\n` envelope and a
// 5000 ms timeout. On transport success the HTTP status code is
// written to stdout as decimal+newline; on any transport failure the
// literal string `error\n` is written instead. Exit is always 0.
pub fun main(world: World) -> Void raises {
let maybe_url = std.args.get(1)
if maybe_url.has == false {
check world.out.write("error\n")
return
}
let url_arg = maybe_url.value
// Build the request envelope: "GET <url>\n\n" in a fixed buffer.
// 256 bytes URL cap + 6 bytes overhead leaves room comfortably.
let mut envelope: [320]u8 = [0_u8; 320]
let mut env_n: usize = 0
envelope[0] = 71_u8
envelope[1] = 69_u8
envelope[2] = 84_u8
envelope[3] = 32_u8
env_n = 4
let mut i: usize = 0
let url_n = std.mem.len(url_arg)
while i < url_n {
let b = url_arg[i]
if b == 10_u8 {
i = url_n
} else {
if b == 13_u8 {
i = url_n
} else {
envelope[env_n] = b
env_n = env_n + 1
i = i + 1
}
}
}
envelope[env_n] = 10_u8
env_n = env_n + 1
envelope[env_n] = 10_u8
env_n = env_n + 1
let net = std.net.host()
let client = std.http.client(net)
let mut response: [8192]u8 = [0_u8; 8192]
let result = std.http.fetch(client, envelope[0..env_n], response, std.time.ms(5000))
let err = std.http.resultError(result)
let mut out: [32]u8 = [0_u8; 32]
let mut out_n: usize = 0
if err == std.http.errorNone() {
let status = std.http.resultStatus(result)
let mut v: u32 = status as u32
if v == 0_u32 {
out[0] = 48_u8
out_n = 1
} else {
let mut tmp: [16]u8 = [0_u8; 16]
let mut tn: usize = 0
while v > 0_u32 {
tmp[tn] = (48_u32 + (v % 10_u32)) as u8
tn = tn + 1
v = v / 10_u32
}
let mut j: usize = 0
while j < tn {
out[out_n] = tmp[tn - 1 - j]
out_n = out_n + 1
j = j + 1
}
}
out[out_n] = 10_u8
out_n = out_n + 1
} else {
out[0] = 101_u8
out[1] = 114_u8
out[2] = 114_u8
out[3] = 111_u8
out[4] = 114_u8
out[5] = 10_u8
out_n = 6
}
check world.out.write(out[0..out_n])
return
}
TypeScript 25 lines
async function main(): Promise<void> {
const chunks: Buffer[] = [];
for await (const chunk of process.stdin) {
chunks.push(chunk as Buffer);
}
const url = Buffer.concat(chunks).toString("utf-8").trim();
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), 5000);
try {
const response = await fetch(url, {
method: "GET",
signal: controller.signal,
redirect: "manual",
});
process.stdout.write(`${response.status}\n`);
} catch {
process.stdout.write("error\n");
} finally {
clearTimeout(timer);
}
}
main();
Rust 32 lines
use std::io::{self, Read, Write};
use std::time::Duration;
fn main() {
let mut url = String::new();
if io::stdin().read_to_string(&mut url).is_err() {
let _ = io::stdout().write_all(b"error\n");
return;
}
let url = url.trim();
let agent = ureq::Agent::config_builder()
.timeout_global(Some(Duration::from_secs(5)))
.build()
.new_agent();
let result = agent.get(url).call();
let stdout = io::stdout();
let mut handle = stdout.lock();
match result {
Ok(response) => {
let _ = writeln!(handle, "{}", response.status().as_u16());
}
Err(ureq::Error::StatusCode(code)) => {
let _ = writeln!(handle, "{}", code);
}
Err(_) => {
let _ = handle.write_all(b"error\n");
}
}
}
Go 34 lines
package main
import (
"fmt"
"io"
"net/http"
"os"
"strings"
"time"
)
func main() {
urlBytes, err := io.ReadAll(os.Stdin)
if err != nil {
os.Stdout.WriteString("error\n")
return
}
url := strings.TrimSpace(string(urlBytes))
client := &http.Client{
Timeout: 5 * time.Second,
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
}
resp, err := client.Get(url)
if err != nil {
os.Stdout.WriteString("error\n")
return
}
defer resp.Body.Close()
fmt.Printf("%d\n", resp.StatusCode)
}
Python 24 lines
#!/usr/bin/env python3
import sys
import urllib.request
import urllib.error
def main() -> int:
url = sys.stdin.read().strip()
try:
req = urllib.request.Request(url, method="GET")
with urllib.request.urlopen(req, timeout=5) as response:
status = response.status
except urllib.error.HTTPError as exc:
status = exc.code
except (urllib.error.URLError, ValueError, OSError, TimeoutError):
sys.stdout.write("error\n")
return 0
sys.stdout.write(f"{status}\n")
return 0
if __name__ == "__main__":
raise SystemExit(main())
Design notes
Algorithm, failure modes, cross-language parity, and where Zero needed a workaround. From corpus/012-http-status-code/notes.md.
Algorithm
There is no algorithm. The task is direct: read a URL, GET it, write
the HTTP status code as decimal, write error on transport failure.
All references converge on the same shape: read input → call the
language's HTTP client → branch on transport-success-vs-error →
write the decimal status code or the literal error\n.
Fixture
A local Python HTTP server listens on 127.0.0.1:18012 while
verify.sh runs the references. Routes:
/status/<code>returns HTTP<code>with an empty body/and any other path return 200 OK
verify.sh starts the fixture in the background, waits for it to
print ready on stdout, runs all references against six cases, and
kills the fixture on exit. The unresolvable-host case
(http://this-host-does-not-resolve.invalid/) does not touch the
fixture; it exercises every reference's transport-error path.
Edge cases
- Trailing newline on the URL must be trimmed before use.
- The
/status/<code>route is the contract; the references must treat the status code as a number, not as text from a body. - Transport-failure path covers DNS failure, connect-refused, TLS
error, timeout, invalid URL, unsupported protocol, provider
unavailable, and I/O failure. All collapse to
error\n.
Zero-specific notes
- argv[1] carries the URL because Zero 0.1.2 has no exposed stdin.
- The HTTP request envelope for
std.http.fetchisGET <url>\n\nin a[320]u8buffer; the URL is bounded by the 256-byte spec cap so the envelope fits with overhead. - Response buffer is
[8192]u8. The references use only the status field viastd.http.resultStatus, so body and header parsing is not exercised here. - Decimal renderer reuses the small-divisor u32 divmod pattern from prior tasks (avoids the 2^32-divisor SIGFPE).
- The Zero
std.http.fetchprovider is implemented in libcurl and is link-time. On a VM withoutlibcurl4-openssl-dev, the Zero build fails at link time withBLD003: host runtime link failed. Userland fix: extract the deb into~/.local/includeand symlink~/.local/lib/libcurl.so→/usr/lib/x86_64-linux-gnu/libcurl.so.4, thenexport C_INCLUDE_PATH=~/.local/includeandexport LIBRARY_PATH=~/.local/lib. These are wired into~/.config/truffle/env.shfor this machine. - Zero 0.1.2 codegen leaks the trailing
world.out.write(...)return value (bytes written) as the process exit code unless main has an explicitreturnat the end. Symptom: stdout shows200but exit is4. Fix: end main withreturn.
Cross-implementation parity
All five references issue a GET, observe one HTTP transaction, and
write <status>\n on success or error\n on failure. No retries,
no redirects (each reference explicitly opts out of redirect
following), no body inspection. Byte-exact agreement on every case.
Cost
| Model | Prompt tokens | Completion tokens | API ms |
|---|---|---|---|
| gpt-4o | 3,696 | 765 | 7,873 |
| gpt-4o-mini | 3,696 | 697 | 15,462 |
| gpt-5 | 3,691 | 20,565 | 192,124 |
Tokens and API ms are summed across the five languages this model attempted for this task.
Compare
Model deep-dives: gpt-4o · gpt-4o-mini · gpt-5 . Back to the leaderboard and methodology.