GET a URL and write the HTTP status code

Prompt

This is the natural-language brief given to every model, verbatim. The harness prefixes a language-specific calling-convention block and suffixes a "return only the source code" instruction. Nothing else.

# 012-http-status-code

Read a single URL from standard input. Issue an HTTP GET request to
that URL with a 5000 millisecond transport timeout. Write the HTTP
response status code as decimal followed by a newline to standard
output (e.g. `200\n`).

If the transport fails for any reason — DNS lookup failure, connect
refused, TLS error, request timed out, invalid URL, unsupported
protocol, the HTTP provider being unavailable, or any I/O error —
write the literal string `error\n` to standard output instead of a
status code. Do not write to standard error in either case. Exit
with status 0 in every case.

The URL on stdin may have a trailing newline; trim it before use.

## Examples

Input (stdin):

```
http://localhost:18012/status/200
```

Output (stdout):

```
200
```

Input (stdin):

```
http://localhost:18012/status/404
```

Output (stdout):

```
404
```

Input (stdin):

```
http://this-host-does-not-resolve.invalid/
```

Output (stdout):

```
error
```

## Acceptance

- stdout matches the expected bytes exactly per test case
- stderr is empty
- exit code is 0
- the run completes within 10 seconds wall time

## Input convention by language

- **TypeScript / Rust / Go / Python**: read the URL from stdin until
  newline or EOF; trim trailing newline.
- **Zero**: take the URL from `argv[1]` because Zero 0.1.2 does not
  expose a standard-input capability. The byte semantics are the
  same — Zero callers should treat the argv string identically to a
  stdin read.

## Verifier fixture

The verifier starts a local Python HTTP fixture server on port 18012
before running the references and tears it down after. The server
responds to `/status/<code>` with status `<code>` and an empty body,
to `/` with 200 OK, and to any other path with 200 OK. References
target `http://localhost:18012/...` for the happy-path cases. For
the transport-failure cases the URL points at a name that does not
resolve, so the reference must surface the failure as `error\n`.

Acceptance

A task counts as passed only when every public and hidden test case agrees on these fields. No fuzzy matching, no "off by one trailing newline is fine."

Results

Each cell is one attempt. Pass means stdout matched byte-exact on every test case, stderr empty, exit zero. Hover a failure to see the captured first line of the diagnostic.

Failure excerpts

13 of 25 attempts failed. Each card is one attempt, with the captured first line of the diagnostic.

1. gpt-4o Zero compile

   ref.zero:1:1 IMP001: unknown package-local import 'std::net'

2. gpt-4o Rust wrong output

   (no diagnostic captured)

3. gpt-4o Python other

   Traceback (most recent call last):

4. gpt-4o-mini Zero compile

   ref.zero:1:1 IMP001: unknown package-local import 'lib http'

5. gpt-4o-mini TypeScript wrong output

   (no diagnostic captured)

6. gpt-4o-mini Rust wrong output

   (no diagnostic captured)

7. gpt-4o-mini Python other

   Traceback (most recent call last):

8. gpt-5 Zero compile

   ref.zero:1:1 IMP001: unknown package-local import 'std'

9. opus Zero compile

   ref.zero:1:13 PAR100: unexpected character '@'

10. opus Rust wrong output

    (no diagnostic captured)

11. sonnet Zero wrong output

    (no diagnostic captured)

12. sonnet Rust wrong output

    (no diagnostic captured)

13. sonnet Python other

    (no diagnostic captured)

Reference implementations

The hand-written reference each language ships with. Every reference passes the same public and hidden test suite under the pinned toolchain before any model touches the task.

Click a language to expand

async function main(): Promise<void> {
  const chunks: Buffer[] = [];
  for await (const chunk of process.stdin) {
    chunks.push(chunk as Buffer);
  }
  const url = Buffer.concat(chunks).toString("utf-8").trim();

  const controller = new AbortController();
  const timer = setTimeout(() => controller.abort(), 5000);
  try {
    const response = await fetch(url, {
      method: "GET",
      signal: controller.signal,
      redirect: "manual",
    });
    process.stdout.write(`${response.status}\n`);
  } catch {
    process.stdout.write("error\n");
  } finally {
    clearTimeout(timer);
  }
}

main();

use std::io::{self, Read, Write};
use std::time::Duration;

fn main() {
    let mut url = String::new();
    if io::stdin().read_to_string(&mut url).is_err() {
        let _ = io::stdout().write_all(b"error\n");
        return;
    }
    let url = url.trim();

    let agent = ureq::Agent::config_builder()
        .timeout_global(Some(Duration::from_secs(5)))
        .build()
        .new_agent();

    let result = agent.get(url).call();
    let stdout = io::stdout();
    let mut handle = stdout.lock();
    match result {
        Ok(response) => {
            let _ = writeln!(handle, "{}", response.status().as_u16());
        }
        Err(ureq::Error::StatusCode(code)) => {
            let _ = writeln!(handle, "{}", code);
        }
        Err(_) => {
            let _ = handle.write_all(b"error\n");
        }
    }
}

package main

import (
	"fmt"
	"io"
	"net/http"
	"os"
	"strings"
	"time"
)

func main() {
	urlBytes, err := io.ReadAll(os.Stdin)
	if err != nil {
		os.Stdout.WriteString("error\n")
		return
	}
	url := strings.TrimSpace(string(urlBytes))

	client := &http.Client{
		Timeout: 5 * time.Second,
		CheckRedirect: func(req *http.Request, via []*http.Request) error {
			return http.ErrUseLastResponse
		},
	}
	resp, err := client.Get(url)
	if err != nil {
		os.Stdout.WriteString("error\n")
		return
	}
	defer resp.Body.Close()
	fmt.Printf("%d\n", resp.StatusCode)
}

#!/usr/bin/env python3
import sys
import urllib.request
import urllib.error


def main() -> int:
    url = sys.stdin.read().strip()
    try:
        req = urllib.request.Request(url, method="GET")
        with urllib.request.urlopen(req, timeout=5) as response:
            status = response.status
    except urllib.error.HTTPError as exc:
        status = exc.code
    except (urllib.error.URLError, ValueError, OSError, TimeoutError):
        sys.stdout.write("error\n")
        return 0
    sys.stdout.write(f"{status}\n")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Design notes

Algorithm, failure modes, cross-language parity, and where Zero needed a workaround. From corpus/012-http-status-code/notes.md.

Cost

Tokens and API ms are summed across the five languages this model attempted for this task.

Compare

Model deep-dives: gpt-4o · gpt-4o-mini · gpt-5 · opus · sonnet . Back to the leaderboard and methodology.

Truffle · 012-http-status-code · run captured 2026-06-27