agentlang-index · task medium

GET a URL and write the HTTP status code

012-http-status-code. Read a single URL from standard input.

Prompt

This is the natural-language brief given to every model, verbatim. The harness prefixes a language-specific calling-convention block and suffixes a "return only the source code" instruction. Nothing else.

# 012-http-status-code

Read a single URL from standard input. Issue an HTTP GET request to
that URL with a 5000 millisecond transport timeout. Write the HTTP
response status code as decimal followed by a newline to standard
output (e.g. `200\n`).

If the transport fails for any reason — DNS lookup failure, connect
refused, TLS error, request timed out, invalid URL, unsupported
protocol, the HTTP provider being unavailable, or any I/O error —
write the literal string `error\n` to standard output instead of a
status code. Do not write to standard error in either case. Exit
with status 0 in every case.

The URL on stdin may have a trailing newline; trim it before use.

## Examples

Input (stdin):

```
http://localhost:18012/status/200
```

Output (stdout):

```
200
```

Input (stdin):

```
http://localhost:18012/status/404
```

Output (stdout):

```
404
```

Input (stdin):

```
http://this-host-does-not-resolve.invalid/
```

Output (stdout):

```
error
```

## Acceptance

- stdout matches the expected bytes exactly per test case
- stderr is empty
- exit code is 0
- the run completes within 10 seconds wall time

## Input convention by language

- **TypeScript / Rust / Go / Python**: read the URL from stdin until
  newline or EOF; trim trailing newline.
- **Zero**: take the URL from `argv[1]` because Zero 0.1.2 does not
  expose a standard-input capability. The byte semantics are the
  same — Zero callers should treat the argv string identically to a
  stdin read.

## Verifier fixture

The verifier starts a local Python HTTP fixture server on port 18012
before running the references and tears it down after. The server
responds to `/status/<code>` with status `<code>` and an empty body,
to `/` with 200 OK, and to any other path with 200 OK. References
target `http://localhost:18012/...` for the happy-path cases. For
the transport-failure cases the URL points at a name that does not
resolve, so the reference must surface the failure as `error\n`.

Acceptance

A task counts as passed only when every public and hidden test case agrees on these fields. No fuzzy matching, no "off by one trailing newline is fine."

stdout (byte-exact, per case) true
stderr (exact bytes) ""
exit code 0
wall time max (ms) 10000
tags networking, http, stdlib-breadth

Results

Each cell is one attempt. Pass means stdout matched byte-exact on every test case, stderr empty, exit zero. Hover a failure to see the captured first line of the diagnostic.

Model ZeroTypeScriptRustGoPython
gpt-4o compile wrong output other
gpt-4o-mini compile wrong output wrong output other
gpt-5 compile

Failure excerpts

8 of 15 attempts failed. Each card is one attempt, with the captured first line of the diagnostic.

  1. gpt-4o Zero compile
    ref.zero:1:1 IMP001: unknown package-local import 'std::net'
  2. gpt-4o Rust wrong output
    (no diagnostic captured)
  3. gpt-4o Python other
    Traceback (most recent call last):
  4. gpt-4o-mini Zero compile
    ref.zero:1:1 IMP001: unknown package-local import 'lib http'
  5. gpt-4o-mini TypeScript wrong output
    (no diagnostic captured)
  6. gpt-4o-mini Rust wrong output
    (no diagnostic captured)
  7. gpt-4o-mini Python other
    Traceback (most recent call last):
  8. gpt-5 Zero compile
    ref.zero:1:1 IMP001: unknown package-local import 'std'

Reference implementations

The hand-written reference each language ships with. Every reference passes the same public and hidden test suite under the pinned toolchain before any model touches the task.

Click a language to expand

Zero 90 lines
// HTTP GET status-code reference for AgentLang Index.
//
// Zero 0.1.2 has no exposed stdin, so the URL is read from argv[1].
// std.http.fetch is invoked with a `GET <url>\n\n` envelope and a
// 5000 ms timeout. On transport success the HTTP status code is
// written to stdout as decimal+newline; on any transport failure the
// literal string `error\n` is written instead. Exit is always 0.

pub fun main(world: World) -> Void raises {
    let maybe_url = std.args.get(1)
    if maybe_url.has == false {
        check world.out.write("error\n")
        return
    }
    let url_arg = maybe_url.value

    // Build the request envelope: "GET <url>\n\n" in a fixed buffer.
    // 256 bytes URL cap + 6 bytes overhead leaves room comfortably.
    let mut envelope: [320]u8 = [0_u8; 320]
    let mut env_n: usize = 0
    envelope[0] = 71_u8
    envelope[1] = 69_u8
    envelope[2] = 84_u8
    envelope[3] = 32_u8
    env_n = 4
    let mut i: usize = 0
    let url_n = std.mem.len(url_arg)
    while i < url_n {
        let b = url_arg[i]
        if b == 10_u8 {
            i = url_n
        } else {
            if b == 13_u8 {
                i = url_n
            } else {
                envelope[env_n] = b
                env_n = env_n + 1
                i = i + 1
            }
        }
    }
    envelope[env_n] = 10_u8
    env_n = env_n + 1
    envelope[env_n] = 10_u8
    env_n = env_n + 1

    let net = std.net.host()
    let client = std.http.client(net)
    let mut response: [8192]u8 = [0_u8; 8192]
    let result = std.http.fetch(client, envelope[0..env_n], response, std.time.ms(5000))
    let err = std.http.resultError(result)

    let mut out: [32]u8 = [0_u8; 32]
    let mut out_n: usize = 0
    if err == std.http.errorNone() {
        let status = std.http.resultStatus(result)
        let mut v: u32 = status as u32
        if v == 0_u32 {
            out[0] = 48_u8
            out_n = 1
        } else {
            let mut tmp: [16]u8 = [0_u8; 16]
            let mut tn: usize = 0
            while v > 0_u32 {
                tmp[tn] = (48_u32 + (v % 10_u32)) as u8
                tn = tn + 1
                v = v / 10_u32
            }
            let mut j: usize = 0
            while j < tn {
                out[out_n] = tmp[tn - 1 - j]
                out_n = out_n + 1
                j = j + 1
            }
        }
        out[out_n] = 10_u8
        out_n = out_n + 1
    } else {
        out[0] = 101_u8
        out[1] = 114_u8
        out[2] = 114_u8
        out[3] = 111_u8
        out[4] = 114_u8
        out[5] = 10_u8
        out_n = 6
    }
    check world.out.write(out[0..out_n])
    return
}
TypeScript 25 lines
async function main(): Promise<void> {
  const chunks: Buffer[] = [];
  for await (const chunk of process.stdin) {
    chunks.push(chunk as Buffer);
  }
  const url = Buffer.concat(chunks).toString("utf-8").trim();

  const controller = new AbortController();
  const timer = setTimeout(() => controller.abort(), 5000);
  try {
    const response = await fetch(url, {
      method: "GET",
      signal: controller.signal,
      redirect: "manual",
    });
    process.stdout.write(`${response.status}\n`);
  } catch {
    process.stdout.write("error\n");
  } finally {
    clearTimeout(timer);
  }
}

main();
Rust 32 lines
use std::io::{self, Read, Write};
use std::time::Duration;

fn main() {
    let mut url = String::new();
    if io::stdin().read_to_string(&mut url).is_err() {
        let _ = io::stdout().write_all(b"error\n");
        return;
    }
    let url = url.trim();

    let agent = ureq::Agent::config_builder()
        .timeout_global(Some(Duration::from_secs(5)))
        .build()
        .new_agent();

    let result = agent.get(url).call();
    let stdout = io::stdout();
    let mut handle = stdout.lock();
    match result {
        Ok(response) => {
            let _ = writeln!(handle, "{}", response.status().as_u16());
        }
        Err(ureq::Error::StatusCode(code)) => {
            let _ = writeln!(handle, "{}", code);
        }
        Err(_) => {
            let _ = handle.write_all(b"error\n");
        }
    }
}
Go 34 lines
package main

import (
	"fmt"
	"io"
	"net/http"
	"os"
	"strings"
	"time"
)

func main() {
	urlBytes, err := io.ReadAll(os.Stdin)
	if err != nil {
		os.Stdout.WriteString("error\n")
		return
	}
	url := strings.TrimSpace(string(urlBytes))

	client := &http.Client{
		Timeout: 5 * time.Second,
		CheckRedirect: func(req *http.Request, via []*http.Request) error {
			return http.ErrUseLastResponse
		},
	}
	resp, err := client.Get(url)
	if err != nil {
		os.Stdout.WriteString("error\n")
		return
	}
	defer resp.Body.Close()
	fmt.Printf("%d\n", resp.StatusCode)
}
Python 24 lines
#!/usr/bin/env python3
import sys
import urllib.request
import urllib.error


def main() -> int:
    url = sys.stdin.read().strip()
    try:
        req = urllib.request.Request(url, method="GET")
        with urllib.request.urlopen(req, timeout=5) as response:
            status = response.status
    except urllib.error.HTTPError as exc:
        status = exc.code
    except (urllib.error.URLError, ValueError, OSError, TimeoutError):
        sys.stdout.write("error\n")
        return 0
    sys.stdout.write(f"{status}\n")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Design notes

Algorithm, failure modes, cross-language parity, and where Zero needed a workaround. From corpus/012-http-status-code/notes.md.

Algorithm

There is no algorithm. The task is direct: read a URL, GET it, write the HTTP status code as decimal, write error on transport failure. All references converge on the same shape: read input → call the language's HTTP client → branch on transport-success-vs-error → write the decimal status code or the literal error\n.

Fixture

A local Python HTTP server listens on 127.0.0.1:18012 while verify.sh runs the references. Routes:

  • /status/<code> returns HTTP <code> with an empty body
  • / and any other path return 200 OK

verify.sh starts the fixture in the background, waits for it to print ready on stdout, runs all references against six cases, and kills the fixture on exit. The unresolvable-host case (http://this-host-does-not-resolve.invalid/) does not touch the fixture; it exercises every reference's transport-error path.

Edge cases

  • Trailing newline on the URL must be trimmed before use.
  • The /status/<code> route is the contract; the references must treat the status code as a number, not as text from a body.
  • Transport-failure path covers DNS failure, connect-refused, TLS error, timeout, invalid URL, unsupported protocol, provider unavailable, and I/O failure. All collapse to error\n.

Zero-specific notes

  • argv[1] carries the URL because Zero 0.1.2 has no exposed stdin.
  • The HTTP request envelope for std.http.fetch is GET <url>\n\n in a [320]u8 buffer; the URL is bounded by the 256-byte spec cap so the envelope fits with overhead.
  • Response buffer is [8192]u8. The references use only the status field via std.http.resultStatus, so body and header parsing is not exercised here.
  • Decimal renderer reuses the small-divisor u32 divmod pattern from prior tasks (avoids the 2^32-divisor SIGFPE).
  • The Zero std.http.fetch provider is implemented in libcurl and is link-time. On a VM without libcurl4-openssl-dev, the Zero build fails at link time with BLD003: host runtime link failed. Userland fix: extract the deb into ~/.local/include and symlink ~/.local/lib/libcurl.so/usr/lib/x86_64-linux-gnu/libcurl.so.4, then export C_INCLUDE_PATH=~/.local/include and export LIBRARY_PATH=~/.local/lib. These are wired into ~/.config/truffle/env.sh for this machine.
  • Zero 0.1.2 codegen leaks the trailing world.out.write(...) return value (bytes written) as the process exit code unless main has an explicit return at the end. Symptom: stdout shows 200 but exit is 4. Fix: end main with return.

Cross-implementation parity

All five references issue a GET, observe one HTTP transaction, and write <status>\n on success or error\n on failure. No retries, no redirects (each reference explicitly opts out of redirect following), no body inspection. Byte-exact agreement on every case.


Cost

Model Prompt tokens Completion tokens API ms
gpt-4o 3,696 765 7,873
gpt-4o-mini 3,696 697 15,462
gpt-5 3,691 20,565 192,124

Tokens and API ms are summed across the five languages this model attempted for this task.


Compare

Model deep-dives: gpt-4o · gpt-4o-mini · gpt-5 . Back to the leaderboard and methodology.