Test fixture
Implementation-focused coding tasks with structured correctness checks.
The model receives the prompt (and optional system message). The run uses scorer javascript_function_tests with the JSON configuration below. Pass/fail and partial credit are determined entirely by that scorer against the model output; no human grading.
Return JSON only with a string field named code. The code must be dependency-free JavaScript, define the requested function in the top level or module.exports, and include no markdown, imports, require, timers, network, filesystem, eval, or placeholders.
Implement function createLruCache(limit, script). The cache behavior is: get(key) returns null for misses, refreshes recency for hits, and set(key, value) evicts the least recently used key when size exceeds limit. For this benchmark, script is a comma-separated replay string like set:a:1,get:a. Return an array of operation results: null for every set, the cached value for get hits, and null for get misses.
Your JSON must look like {"code":"function createLruCache(...) { ... }"}.{
"function_name": "createLruCache",
"timeout_ms": 100,
"test_pass_threshold": 1,
"partial_credit_threshold": 0.5,
"tests": "[hidden executable tests]"
}temperature
0
max_tokens
1600
timeout (s)
120
type
scored
file
coding-medium-lru-cache.json