# Introducing SolidWorks Bench

> SolidWorks Bench evaluates frontier AI models on 73 machines, each modelled and assembled in SolidWorks from its drawing package.

URL: https://www.markovstudios.com/research/solidworks-bench

Index of every page, for language models and agents: https://www.markovstudios.com/llms.txt

Benchmark · 17 Nov 2026

Benchmarking how well frontier models perform real mechanical designing tasks in SolidWorks.

### Mean reward

73 machines · PNG drawings · one attempt

| Model | Mean reward |
| --- | --- |
| Fable 5.2 | 0.573 |
| GPT-6 Astra | 0.426 |
| Sonnet 5.5 | 0.347 |
| Opus 5.5 | 0.342 |
| Kimi K3.1 | 0.196 |
| DeepSeek V4.1 | 0.127 |
| Muse Spark | 0.073 |
| GLM 5.3 | 0.064 |
| MiniMax M3.1 | 0.024 |

## 01 Presenting SolidWorks Bench

SolidWorks Bench tests whether multimodal agents can do a mechanical designer's job end to end inside SolidWorks. Each task is a real machine (a robot, a 3D printer, a CNC router) given only as an engineering drawing package: a general-arrangement drawing, one fully dimensioned drawing per custom part, and the supplier STEP files of the bought parts. The agent must model every custom part, assemble the machine with mates that hold it together and let it move, and save native files an engineer could pick up and edit.

No model completes a single machine. The best, Fable 5.2, averages 0.57 on our reward, and no model gets more than 43% of its custom parts exactly right (Fable 5.2).

73 machines

3,434 custom parts to model

5,959 bought parts to place

12,213 drawing sheets

## 02 The tasks

### Task size

| Per machine | Min | Median | Max | Total |
| --- | --- | --- | --- | --- |
| Custom parts to model | 4 | 41 | 185 | 3,434 |
| Bought parts (STEP supplied) | 6 | 57 | 731 | 5,959 |
| Components in the finished machine | 90 | 433 | 2,175 | 36,983 |
| Drawing sheets | 24 | 117 | 1,281 | 12,213 |
| Time limit | 2.7 h | 9.0 h | 24 h | 731 h |

### Machines

| Category | Machines | Median parts |
| --- | --- | --- |
| 3D printers, CNC and lasers | 12 | 44 |
| Aircraft, drones and rockets | 6 | 38 |
| Arms, legs and robot cells | 9 | 34 |
| Mobile robots and vehicles | 24 | 53 |
| Products and equipment | 11 | 32 |
| Tooling, fixtures and test rigs | 11 | 42 |

### Expected output

Native files

One part file per custom part (feature tree, defined sketches, hole features, materials) and the top assembly with its sub-assemblies, mated so the machine holds together and its joints move.

Neutral export

A STEP of the finished machine, which the geometry grader reads. It must be what the native files produce.

## 03 The harness

Each turn the model sees one fresh screenshot, emits an ordered batch of actions, and gets the screen back. Besides the mouse and keyboard it may script SolidWorks through its API in Python. A rollout ends when the model calls finish, or at the turn or time limit.

[Image: Diagram of the SolidWorks harness showing the task inputs, the model and its tools, its ordered action output, SolidWorks on a Windows host, and screenshot feedback]

The loop ends when the model has saved its parts, assemblies and A000.STEP and calls finish, or at the turn or time limit.

## 04 The sandbox

Software

SolidWorks 2026 on Windows, one fresh host per rollout.

Inputs

Drawings as raster PNG only. With vector PDFs, models read coordinates out of the file instead of reading the dimensions.

Isolation

No internet.

Limits

Each machine's own time limit (2.7 to 24 h), 2,000 turns, one attempt, reasoning effort high for every model.

Artifacts

Native files, STEP, every screenshot, model turn and action are kept for audit.

## 05 Scoring

Geometry is read from the STEP in a Linux container. Everything a STEP cannot carry (mates, motion, feature trees, materials) is read from the native files by SolidWorks itself on a separate grading host.

reward = gate × native files × parity × validity × Σ weight × component

gate

the STEP exists and holds a solid

parity

the STEP is what the native files produce

validity

every part is a closed solid

| Component | Weight | Checks |
| --- | --- | --- |
| Geometry | 45% | |
| Parts | 25% | Every face of each modelled part checked against the reference at 0.025 mm and 0.1°, plus the exact volume difference |
| Placement | 20% | Every pair of parts that touch in the reference, graded on how the two sit against each other |
| Assembly function | 30% | |
| Ties | 10% | Each component mated to a neighbour it really touches, weighted by how well it sits |
| Motion | 10% | SolidWorks drags each joint: hinges turn about the right axis, slides slide, rigid parts hold |
| Lock-free | 5% | No Lock-mate shortcuts |
| Patterns | 5% | Repeated components placed by patterns |
| Part practice | 25% | |
| Flex | 6% | A key dimension changed by 5 to 30%, rebuilt, and compared with the reference changed the same way |
| Materials | 7% | The material from the bill of materials on each part |
| Sketches | 4% | Fully defined without Fix relations |
| Holes | 4% | Hole features where the drawing calls for them |
| Part patterns | 4% | Repeated features patterned, not copied |

Practice credit is weighted part by part by that part's geometry, and mating credit by how well the part sits: good technique on a wrong part earns little. A machine is passed only when every part and joint is exact and every check is clean.

## 06 Results

Fable 5.2 leads by a wide margin (0.57 against GPT-6 Astra's 0.43): it builds the best parts and mates its machines as well. Behind it, GPT-6 Astra leads because it mates and patterns its assemblies, and the other Claude models build better parts.

### Mean reward

73 machines · PNG drawings · one attempt

| Model | Mean reward |
| --- | --- |
| Fable 5.2 | 0.573 |
| GPT-6 Astra | 0.426 |
| Sonnet 5.5 | 0.347 |
| Opus 5.5 | 0.342 |
| Kimi K3.1 | 0.196 |
| DeepSeek V4.1 | 0.127 |
| Muse Spark | 0.073 |
| GLM 5.3 | 0.064 |
| MiniMax M3.1 | 0.024 |

### Where the reward comes from

Geometry · Assembly · Part practice

| Model | Geometry 45% | Assembly 30% | Practice 25% |
| --- | --- | --- | --- |
| Fable 5.2 | 0.64 | 0.51 | 0.56 |
| GPT-6 Astra | 0.40 | 0.54 | 0.34 |
| Sonnet 5.5 | 0.39 | 0.32 | 0.30 |
| Opus 5.5 | 0.49 | 0.09 | 0.37 |
| Kimi K3.1 | 0.14 | 0.37 | 0.12 |
| DeepSeek V4.1 | 0.06 | 0.32 | 0.06 |
| Muse Spark | 0.04 | 0.24 | 0.02 |
| GLM 5.3 | 0.08 | 0.27 | 0.06 |
| MiniMax M3.1 | 0.06 | 0.30 | 0.05 |

The Claude models build the best parts; GPT-6 Astra mates its machines (assembly 0.54) where Opus 5.5 pins parts in place (0.09).

### Every graded component

11 components · mean over gradable trials

| Model | Parts | Placement | Ties | Motion | Lock-free | Patterns | Sketches | Holes | Part patterns | Materials | Flex |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Fable 5.2 | 0.73 | 0.53 | 0.34 | 0.46 | 1.00 | 0.45 | 0.71 | 0.58 | 0.39 | 0.61 | 0.43 |
| GPT-6 Astra | 0.53 | 0.24 | 0.29 | 0.44 | 1.00 | 0.93 | 0.47 | 0.43 | 0.20 | 0.38 | 0.24 |
| Sonnet 5.5 | 0.57 | 0.17 | 0.09 | 0.34 | 0.84 | 0.14 | 0.38 | 0.17 | 0.06 | 0.34 | 0.38 |
| Opus 5.5 | 0.65 | 0.28 | 0.07 | 0.07 | 0.21 | 0.00 | 0.49 | 0.19 | 0.08 | 0.57 | 0.26 |
| Kimi K3.1 | 0.22 | 0.04 | 0.10 | 0.39 | 1.00 | 0.10 | 0.13 | 0.11 | 0.02 | 0.16 | 0.10 |
| DeepSeek V4.1 | 0.09 | 0.03 | 0.09 | 0.32 | 0.98 | 0.00 | 0.02 | 0.01 | 0.00 | 0.06 | 0.17 |
| Muse Spark | 0.06 | 0.02 | 0.07 | 0.14 | 0.91 | 0.00 | 0.03 | 0.00 | 0.00 | 0.02 | 0.02 |
| GLM 5.3 | 0.12 | 0.03 | 0.11 | 0.24 | 0.80 | 0.00 | 0.03 | 0.00 | 0.00 | 0.04 | 0.18 |
| MiniMax M3.1 | 0.08 | 0.03 | 0.17 | 0.14 | 1.00 | 0.00 | 0.02 | 0.01 | 0.00 | 0.05 | 0.13 |

Parts: every face of each custom part within 0.025 mm. Placement: each touching pair of parts on its relative pose. Ties: mates to real neighbours. Motion: joints dragged in SolidWorks. Lock-free, patterns: no Lock mates, repeats patterned. Sketches, holes, part patterns, materials: modelling practice, each weighted by the part's own geometry. Flex: rebuild after a key dimension changes.

## 07 Findings

### The best models assemble machines in opposite ways

The strongest models build assemblies in different ways. Opus 5.5 fixes most parts in space where they belong: the geometry is close, but little is mated and little moves (ties 0.07, motion 0.07, lock-free 0.21). GPT-6 Astra mates its parts and patterns repeated ones (ties 0.29, motion 0.44, lock-free 1.00, patterns 0.93) but places them less precisely (placement 0.24). Fable 5.2 does both: the best parts (0.73) and by far the best placement (0.53, against Opus 5.5's 0.28), with GPT-6 Astra-style mating (ties 0.34, motion 0.46, lock-free 1.00). Only in patterning repeated parts does it trail GPT-6 Astra (0.45 against 0.93).

### Machines look right, but few parts are exact

Most machines the frontier models deliver are recognisable: 95% of GPT-6 Astra's score above 0.25. Very few parts are exact. The most common misses are angled faces and chamfers, hole sizes and positions, and lengths a fraction of a millimetre off. A part that looks right on the screen often fails face by face.

### The API became the primary interface

Every model works mostly through SolidWorks' API: Python and PowerShell scripts make up 46% to 72% of all actions. Clicks and keystrokes are a small minority, used to dismiss dialogs and check the screen. How closely models read the drawings differs tenfold: median drawing views per rollout Fable 5.2 258, Opus 5.5 201, GPT-6 Astra 79, Muse Spark 26.

### More time helps only a little

GPT-6 Astra takes a median 124 turns in 1.4 h; Opus 5.5 398 in 2.3 h; Fable 5.2 363 in 3.6 h (none at its longer time limit); MiniMax M3.1 1447 in 8.2 h. Longer helps only a little: within each model, reward and turns correlate weakly to moderately (Spearman ρ from -0.07 to 0.49).

## 08 Cost and efficiency

| Model | $ / machine | $ all 73 | Median turns | Median time | Time limit hit | Reward |
| --- | --- | --- | --- | --- | --- | --- |
| Fable 5.2 | $69.10 | $5,045 | 363 | 3.6 h | 0 | 0.573 |
| GPT-6 Astra | $47.19 | $3,445 | 124 | 1.4 h | 1 | 0.426 |
| Sonnet 5.5 | $17.47 | $1,275 | 409 | 2.6 h | 0 | 0.347 |
| Opus 5.5 | $24.70 | $1,803 | 398 | 2.3 h | 0 | 0.342 |
| Kimi K3.1 | $10.77 | $786 | 288 | 3.0 h | 3 | 0.196 |
| DeepSeek V4.1 | $1.12 | $82 | 373 | 3.5 h | 5 | 0.127 |
| Muse Spark | $12.98 | $947 | 250 | 3.5 h | 5 | 0.073 |
| GLM 5.3 | $3.04 | $222 | 738 | 6.7 h | 34 | 0.064 |
| MiniMax M3.1 | $15.49 | $1,131 | 1447 | 8.2 h | 55 | 0.024 |

### Reward against cost

API list price per machine · mean reward

API cost from each rollout's own token counts at list prices. The SolidWorks host, a Windows machine per rollout, comes on top.

## 09 Gallery

Some of the most complex machines in the set, each next to what the strongest models built from the same drawings.

### Omni-Wheel Ball-Hitting Competition Robot

Volleyball robot R2 · 94 parts to model · 149 bought · 2,069 components

| Model | Geometry | Parts exact | Joints ok | Placed | Time | Notes |
| --- | --- | --- | --- | --- | --- | --- |
| Fable 5.2 | 0.64 | 70/94 | 1,335 | 2,069 | 6.2 h | Placed 2,069 of 2,069 components where a reference part sits; 70 of 94 custom parts exactly right (mean part score 0.90); 1,335 of 5,907 joints within tolerance. Most misses: angled faces and chamfers; hole sizes and positions; worst, P227 with a face 6 mm out. |
| Opus 5.5 | 0.59 | 53/94 | 962 | 2,069 | 4.3 h | Placed 2,069 of 2,069 components where a reference part sits; 53 of 94 custom parts exactly right (mean part score 0.87); 962 of 5,907 joints within tolerance; 1 extra components. Most misses: angled faces and chamfers; hole sizes and positions; worst, P227 with a face 27 mm out. |
| GPT-6 Astra | 0.45 | 27/94 | 388 | 2,069 | 1.8 h | Placed 2,069 of 2,069 components where a reference part sits; 27 of 94 custom parts exactly right (mean part score 0.65); 388 of 5,907 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P217 with a face 17 mm out. |
| Sonnet 5.5 | 0.40 | 33/94 | 4 | 2,069 | 3.0 h | Placed 2,069 of 2,069 components where a reference part sits; 33 of 94 custom parts exactly right (mean part score 0.69); 4 of 5,907 joints within tolerance. Most misses: flat faces (lengths, thicknesses); hole sizes and positions; worst, P201 with a face 120 mm out. |
| Kimi K3.1 | 0.29 | 14/94 | 137 | 2,037 | 5.3 h | Placed 2,037 of 2,069 components where a reference part sits; 14 of 94 custom parts exactly right (mean part score 0.33); 137 of 5,907 joints within tolerance. Most misses: flat faces (lengths, thicknesses); hole sizes and positions; worst, P182 with a face 110 mm out. |
| DeepSeek V4.1 | 0.10 | 0/94 | 147 | 2,065 | 5.6 h | Placed 2,065 of 2,069 components where a reference part sits; 0 of 94 custom parts exactly right (mean part score 0.00); 147 of 5,907 joints within tolerance. Most misses: flat faces (lengths, thicknesses); hole sizes and positions. |
| GLM 5.3 | 0.08 | 4/94 | 0 | 2,069 | 6.7 h | Placed 2,069 of 2,069 components where a reference part sits; 4 of 94 custom parts exactly right (mean part score 0.15); 0 of 5,907 joints within tolerance. Most misses: flat faces (lengths, thicknesses); hole sizes and positions; worst, P182 with a face 139 mm out. |
| MiniMax M3.1 | 0.00 | 0/94 | 0 | 1 | 12.7 h | Placed 1 of 2,069 components where a reference part sits; 0 of 94 custom parts exactly right (mean part score 0.00); 0 of 5,907 joints within tolerance. |

### Autonomous Quadcopter with Worm-Drive Gripper

Quadcopter with worm-drive gripper · 38 parts to model · 731 bought · 2,175 components

| Model | Geometry | Parts exact | Joints ok | Placed | Time | Notes |
| --- | --- | --- | --- | --- | --- | --- |
| Fable 5.2 | 0.73 | 26/38 | 2,040 | 2,175 | 4.0 h | Placed 2,175 of 2,175 components where a reference part sits; 26 of 38 custom parts exactly right (mean part score 0.82); 2,040 of 5,083 joints within tolerance. Most misses: angled faces and chamfers; flat faces (lengths, thicknesses); worst, P020 with a face 8 mm out. |
| GPT-6 Astra | 0.61 | 18/38 | 1,804 | 2,175 | 4.1 h | Placed 2,175 of 2,175 components where a reference part sits; 18 of 38 custom parts exactly right (mean part score 0.74); 1,804 of 5,083 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P769 with a face 13 mm out. |
| Sonnet 5.5 | 0.57 | 18/38 | 1,370 | 2,175 | 3.9 h | Placed 2,175 of 2,175 components where a reference part sits; 18 of 38 custom parts exactly right (mean part score 0.71); 1,370 of 5,083 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P114 with a face 53 mm out. |
| Opus 5.5 | 0.45 | 22/38 | 0 | 2,175 | 2.1 h | Placed 2,175 of 2,175 components where a reference part sits; 22 of 38 custom parts exactly right (mean part score 0.81); 0 of 5,083 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P769 with a face 39 mm out. |
| Kimi K3.1 | 0.27 | 13/38 | 10 | 2,175 | 7.5 h | Placed 2,175 of 2,175 components where a reference part sits; 13 of 38 custom parts exactly right (mean part score 0.48); 10 of 5,083 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P027 with a face 44 mm out. |
| DeepSeek V4.1 | 0.16 | 0/38 | 1,073 | 2,147 | 7.4 h | Placed 2,147 of 2,175 components where a reference part sits; 0 of 38 custom parts exactly right (mean part score 0.00); 1,073 of 5,083 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers. |
| MiniMax M3.1 | 0.11 | 0/38 | 169 | 2,175 | 17.6 h | Placed 2,175 of 2,175 components where a reference part sits; 0 of 38 custom parts exactly right (mean part score 0.00); 169 of 5,083 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers. |
| GLM 5.3 | 0.00 | – | – | – | 18.2 h | Ran out of time after 18.2 h without exporting the assembly STEP: nothing to grade. |
| Muse Spark | 0.00 | – | – | – | 4.6 h | The delivered assembly STEP does not read (ValueError). |

### Gantry CNC Machine with Vacuum Table

CNC router · 56 parts to model · 109 bought · 1,314 components

| Model | Geometry | Parts exact | Joints ok | Placed | Time | Notes |
| --- | --- | --- | --- | --- | --- | --- |
| Fable 5.2 | 0.76 | 40/56 | 605 | 1,314 | 4.7 h | Placed 1,314 of 1,314 components where a reference part sits; 40 of 56 custom parts exactly right (mean part score 0.91); 605 of 2,638 joints within tolerance. Most misses: angled faces and chamfers; flat faces (lengths, thicknesses); worst, P005 with a face 77 mm out. |
| Opus 5.5 | 0.44 | 8/56 | 7 | 1,314 | 2.6 h | Placed 1,314 of 1,314 components where a reference part sits; 8 of 56 custom parts exactly right (mean part score 0.74); 7 of 2,638 joints within tolerance. Most misses: angled faces and chamfers; flat faces (lengths, thicknesses); worst, P029 with a face 295 mm out. |
| GPT-6 Astra | 0.34 | 11/56 | 71 | 1,314 | 1.7 h | Placed 1,314 of 1,314 components where a reference part sits; 11 of 56 custom parts exactly right (mean part score 0.52); 71 of 2,638 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P029 with a face 287 mm out. |
| Sonnet 5.5 | 0.28 | 6/56 | 9 | 1,314 | 3.6 h | Placed 1,314 of 1,314 components where a reference part sits; 6 of 56 custom parts exactly right (mean part score 0.47); 9 of 2,638 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P029 with a face 287 mm out. |
| Kimi K3.1 | 0.16 | 3/56 | 0 | 1,314 | 3.6 h | Placed 1,314 of 1,314 components where a reference part sits; 3 of 56 custom parts exactly right (mean part score 0.27); 0 of 2,638 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P013 with a face 122 mm out. |
| GLM 5.3 | 0.10 | 0/56 | 0 | 1,314 | 7.8 h | Placed 1,314 of 1,314 components where a reference part sits; 0 of 56 custom parts exactly right (mean part score 0.19); 0 of 2,638 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P080 with a face 199 mm out. |
| MiniMax M3.1 | 0.09 | 1/56 | 2 | 1,277 | 15.8 h | Placed 1,277 of 1,314 components where a reference part sits; 1 of 56 custom parts exactly right (mean part score 0.16); 2 of 2,638 joints within tolerance; 43 extra components. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P029 with a face 196 mm out. |
| Muse Spark | 0.03 | 0/56 | 0 | 1,314 | 3.7 h | Placed 1,314 of 1,314 components where a reference part sits; 0 of 56 custom parts exactly right (mean part score 0.05); 0 of 2,638 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P063 with a face 318 mm out. |
| DeepSeek V4.1 | 0.03 | 0/56 | 0 | 1,314 | 2.8 h | Placed 1,314 of 1,314 components where a reference part sits; 0 of 56 custom parts exactly right (mean part score 0.05); 0 of 2,638 joints within tolerance; 76 extra components. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P063 with a face 290 mm out. |

### Autonomous Competition Robot Main Assembly

Autonomous robot · 130 parts to model · 70 bought · 411 components

| Model | Geometry | Parts exact | Joints ok | Placed | Time | Notes |
| --- | --- | --- | --- | --- | --- | --- |
| Opus 5.5 | 0.28 | 21/130 | 11 | 411 | 2.4 h | Placed 411 of 411 components where a reference part sits; 21 of 130 custom parts exactly right (mean part score 0.44); 11 of 877 joints within tolerance. Most misses: angled faces and chamfers; flat faces (lengths, thicknesses); worst, P049 with a face 56 mm out. |
| Fable 5.2 | 0.27 | 19/130 | 0 | 411 | 2.5 h | Placed 411 of 411 components where a reference part sits; 19 of 130 custom parts exactly right (mean part score 0.48); 0 of 877 joints within tolerance. Most misses: angled faces and chamfers; flat faces (lengths, thicknesses); worst, P046 with a face 52 mm out. |
| Sonnet 5.5 | 0.21 | 13/130 | 0 | 411 | 3.8 h | Placed 411 of 411 components where a reference part sits; 13 of 130 custom parts exactly right (mean part score 0.38); 0 of 877 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P049 with a face 51 mm out. |
| GPT-6 Astra | 0.20 | 11/130 | 0 | 411 | 1.6 h | Placed 411 of 411 components where a reference part sits; 11 of 130 custom parts exactly right (mean part score 0.34); 0 of 877 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P049 with a face 84 mm out. |
| DeepSeek V4.1 | 0.04 | 3/130 | 0 | 411 | 4.5 h | Placed 411 of 411 components where a reference part sits; 3 of 130 custom parts exactly right (mean part score 0.08); 0 of 877 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P166 with a face 71 mm out. |
| GLM 5.3 | 0.03 | 0/130 | 4 | 409 | 2.6 h | Placed 409 of 411 components where a reference part sits; 0 of 130 custom parts exactly right (mean part score 0.03); 4 of 877 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P169 with a face 81 mm out. |
| Kimi K3.1 | 0.02 | 0/130 | 0 | 360 | 2.7 h | Placed 360 of 411 components where a reference part sits; 0 of 130 custom parts exactly right (mean part score 0.03); 0 of 877 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P164 with a face 70 mm out. |
| MiniMax M3.1 | 0.00 | – | – | – | 20.7 h | Ran out of time after 20.7 h without exporting the assembly STEP: nothing to grade. |
| Muse Spark | 0.00 | – | – | – | 1.9 h | The delivered assembly STEP does not read (ValueError). |

### Powder-Bed Inkjet Additive Manufacturing Machine

Binder-jetting 3D printer · 70 parts to model · 67 bought · 687 components

| Model | Geometry | Parts exact | Joints ok | Placed | Time | Notes |
| --- | --- | --- | --- | --- | --- | --- |
| Fable 5.2 | 0.88 | 39/70 | 1,116 | 687 | 5.7 h | Placed 687 of 687 components where a reference part sits; 39 of 70 custom parts exactly right (mean part score 0.90); 1,116 of 1,763 joints within tolerance. Most misses: hole sizes and positions; angled faces and chamfers; worst, P039 with a face 42 mm out. |
| Opus 5.5 | 0.64 | 17/70 | 637 | 687 | 3.5 h | Placed 687 of 687 components where a reference part sits; 17 of 70 custom parts exactly right (mean part score 0.80); 637 of 1,763 joints within tolerance. Most misses: angled faces and chamfers; hole sizes and positions; worst, P086 with a face 175 mm out. |
| Sonnet 5.5 | 0.52 | 11/70 | 134 | 681 | 4.1 h | Placed 681 of 687 components where a reference part sits; 11 of 70 custom parts exactly right (mean part score 0.66); 134 of 1,763 joints within tolerance. Most misses: angled faces and chamfers; hole sizes and positions; worst, P084 with a face 33 mm out. |
| GPT-6 Astra | 0.38 | 7/70 | 81 | 687 | 2.3 h | Placed 687 of 687 components where a reference part sits; 7 of 70 custom parts exactly right (mean part score 0.48); 81 of 1,763 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P084 with a face 49 mm out. |
| GLM 5.3 | 0.06 | 0/70 | 6 | 687 | 4.4 h | Placed 687 of 687 components where a reference part sits; 0 of 70 custom parts exactly right (mean part score 0.11); 6 of 1,763 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P086 with a face 141 mm out. |
| Kimi K3.1 | 0.03 | 0/70 | 0 | 462 | 2.5 h | Placed 462 of 687 components where a reference part sits; 0 of 70 custom parts exactly right (mean part score 0.06); 0 of 1,763 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P006 with a face 170 mm out. |
| DeepSeek V4.1 | 0.03 | 0/70 | 6 | 684 | 3.2 h | Placed 684 of 687 components where a reference part sits; 0 of 70 custom parts exactly right (mean part score 0.04); 6 of 1,763 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P060 with a face 53 mm out. |

### Tracked FRC Competition Robot with Shooter

FRC tracked robot with shooter · 69 parts to model · 73 bought · 564 components

| Model | Geometry | Parts exact | Joints ok | Placed | Time | Notes |
| --- | --- | --- | --- | --- | --- | --- |
| Fable 5.2 | 0.77 | 37/69 | 361 | 564 | 4.8 h | Placed 564 of 564 components where a reference part sits; 37 of 69 custom parts exactly right (mean part score 0.78); 361 of 1,332 joints within tolerance. Most misses: angled faces and chamfers; flat faces (lengths, thicknesses); worst, P085 with a face 7 mm out. |
| Opus 5.5 | 0.66 | 29/69 | 366 | 564 | 4.7 h | Placed 564 of 564 components where a reference part sits; 29 of 69 custom parts exactly right (mean part score 0.67); 366 of 1,332 joints within tolerance. Most misses: angled faces and chamfers; flat faces (lengths, thicknesses); worst, P112 with a face 79 mm out. |
| GPT-6 Astra | 0.35 | 15/69 | 47 | 564 | 1.2 h | Placed 564 of 564 components where a reference part sits; 15 of 69 custom parts exactly right (mean part score 0.45); 47 of 1,332 joints within tolerance. Most misses: angled faces and chamfers; flat faces (lengths, thicknesses); worst, P017 with a face 67 mm out. |
| Sonnet 5.5 | 0.33 | 23/69 | 13 | 564 | 3.3 h | Placed 564 of 564 components where a reference part sits; 23 of 69 custom parts exactly right (mean part score 0.57); 13 of 1,332 joints within tolerance. Most misses: angled faces and chamfers; flat faces (lengths, thicknesses); worst, P017 with a face 63 mm out. |
| Kimi K3.1 | 0.13 | 8/69 | 0 | 557 | 2.1 h | Placed 557 of 564 components where a reference part sits; 8 of 69 custom parts exactly right (mean part score 0.23); 0 of 1,332 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P091 with a face 105 mm out. |
| DeepSeek V4.1 | 0.04 | 0/69 | 1 | 554 | 3.4 h | Placed 554 of 564 components where a reference part sits; 0 of 69 custom parts exactly right (mean part score 0.06); 1 of 1,332 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P091 with a face 132 mm out. |
| Muse Spark | 0.03 | 2/69 | 0 | 562 | 4.2 h | Placed 562 of 564 components where a reference part sits; 2 of 69 custom parts exactly right (mean part score 0.05); 0 of 1,332 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P091 with a face 114 mm out. |
| MiniMax M3.1 | 0.00 | – | – | 0 | 13.3 h | Placed 0 of 564 components where a reference part sits; 0 of 69 custom parts exactly right (mean part score 0.00); 0 of 0 joints within tolerance. |

## Looking for expert CAD computer-use data?

[Talk to us](https://cal.com/dev-markov/30min) [Browse samples](https://www.markovstudios.com/datasets/cad-environments)
