Talk to us
Benchmark

Introducing SolidWorks Bench

Benchmarking how well frontier models perform real mechanical designing tasks in SolidWorks.

Mean reward

73 machines · PNG drawings · one attempt
0.573
0.426
0.347
0.342
0.196
0.127
0.073
0.064
0.024
Fable 5.2GPT-6 AstraSonnet 5.5Opus 5.5Kimi K3.1DeepSeek V4.1Muse SparkGLM 5.3MiniMax M3.1

01Presenting SolidWorks Bench

SolidWorks Bench tests whether multimodal agents can do a mechanical designer's job end to end inside SolidWorks. Each task is a real machine (a robot, a 3D printer, a CNC router) given only as an engineering drawing package: a general-arrangement drawing, one fully dimensioned drawing per custom part, and the supplier STEP files of the bought parts. The agent must model every custom part, assemble the machine with mates that hold it together and let it move, and save native files an engineer could pick up and edit.

No model completes a single machine. The best, Fable 5.2, averages 0.57 on our reward, and no model gets more than 43% of its custom parts exactly right (Fable 5.2).

73machines
3,434custom parts to model
5,959bought parts to place
12,213drawing sheets

02The tasks

Loading sheet…
Swipe or ← → · click to zoom

Task size

Per machineMinMedianMaxTotal
Custom parts to model4411853,434
Bought parts (STEP supplied)6577315,959
Components in the finished machine904332,17536,983
Drawing sheets241171,28112,213
Time limit2.7 h9.0 h24 h731 h

Machines

CategoryMachinesMedian parts
3D printers, CNC and lasers1244
Aircraft, drones and rockets638
Arms, legs and robot cells934
Mobile robots and vehicles2453
Products and equipment1132
Tooling, fixtures and test rigs1142

Expected output

Native files
One part file per custom part (feature tree, defined sketches, hole features, materials) and the top assembly with its sub-assemblies, mated so the machine holds together and its joints move.
Neutral export
A STEP of the finished machine, which the geometry grader reads. It must be what the native files produce.

03The harness

Each turn the model sees one fresh screenshot, emits an ordered batch of actions, and gets the screen back. Besides the mouse and keyboard it may script SolidWorks through its API in Python. A rollout ends when the model calls finish, or at the turn or time limit.

The loop ends when the model has saved its parts, assemblies and A000.STEP and calls finish, or at the turn or time limit.

04The sandbox

Software
SolidWorks 2026 on Windows, one fresh host per rollout.
Inputs
Drawings as raster PNG only. With vector PDFs, models read coordinates out of the file instead of reading the dimensions.
Isolation
No internet.
Limits
Each machine's own time limit (2.7 to 24 h), 2,000 turns, one attempt, reasoning effort high for every model.
Artifacts
Native files, STEP, every screenshot, model turn and action are kept for audit.

05Scoring

Geometry is read from the STEP in a Linux container. Everything a STEP cannot carry (mates, motion, feature trees, materials) is read from the native files by SolidWorks itself on a separate grading host.

reward = gate × native files × parity × validity × Σ weight × component

gate
the STEP exists and holds a solid
parity
the STEP is what the native files produce
validity
every part is a closed solid
ComponentWeightChecks
Geometry45%
Parts25%Every face of each modelled part checked against the reference at 0.025 mm and 0.1°, plus the exact volume difference
Placement20%Every pair of parts that touch in the reference, graded on how the two sit against each other
Assembly function30%
Ties10%Each component mated to a neighbour it really touches, weighted by how well it sits
Motion10%SolidWorks drags each joint: hinges turn about the right axis, slides slide, rigid parts hold
Lock-free5%No Lock-mate shortcuts
Patterns5%Repeated components placed by patterns
Part practice25%
Flex6%A key dimension changed by 5 to 30%, rebuilt, and compared with the reference changed the same way
Materials7%The material from the bill of materials on each part
Sketches4%Fully defined without Fix relations
Holes4%Hole features where the drawing calls for them
Part patterns4%Repeated features patterned, not copied

Practice credit is weighted part by part by that part's geometry, and mating credit by how well the part sits: good technique on a wrong part earns little. A machine is passed only when every part and joint is exact and every check is clean.

06Results

Fable 5.2 leads by a wide margin (0.57 against GPT-6 Astra's 0.43): it builds the best parts and mates its machines as well. Behind it, GPT-6 Astra leads because it mates and patterns its assemblies, and the other Claude models build better parts.

Mean reward

73 machines · PNG drawings · one attempt
0.573
0.426
0.347
0.342
0.196
0.127
0.073
0.064
0.024
Fable 5.2GPT-6 AstraSonnet 5.5Opus 5.5Kimi K3.1DeepSeek V4.1Muse SparkGLM 5.3MiniMax M3.1

Where the reward comes from

Geometry · Assembly · Part practice
Fable 5.2
Geometry 45%
0.64
Assembly 30%
0.51
Practice 25%
0.56
GPT-6 Astra
Geometry 45%
0.40
Assembly 30%
0.54
Practice 25%
0.34
Sonnet 5.5
Geometry 45%
0.39
Assembly 30%
0.32
Practice 25%
0.30
Opus 5.5
Geometry 45%
0.49
Assembly 30%
0.09
Practice 25%
0.37
Kimi K3.1
Geometry 45%
0.14
Assembly 30%
0.37
Practice 25%
0.12
DeepSeek V4.1
Geometry 45%
0.06
Assembly 30%
0.32
Practice 25%
0.06
Muse Spark
Geometry 45%
0.04
Assembly 30%
0.24
Practice 25%
0.02
GLM 5.3
Geometry 45%
0.08
Assembly 30%
0.27
Practice 25%
0.06
MiniMax M3.1
Geometry 45%
0.06
Assembly 30%
0.30
Practice 25%
0.05
The Claude models build the best parts; GPT-6 Astra mates its machines (assembly 0.54) where Opus 5.5 pins parts in place (0.09).

Every graded component

11 components · mean over gradable trials
PartsPlacementTiesMotionLock-freePatternsSketchesHolesPart pat.MaterialsFlexFable 5.20.730.530.340.461.000.450.710.580.390.610.43GPT-6 Astra0.530.240.290.441.000.930.470.430.200.380.24Sonnet 5.50.570.170.090.340.840.140.380.170.060.340.38Opus 5.50.650.280.070.070.210.000.490.190.080.570.26Kimi K3.10.220.040.100.391.000.100.130.110.020.160.10DeepSeek V4.10.090.030.090.320.980.000.020.010.000.060.17Muse Spark0.060.020.070.140.910.000.030.000.000.020.02GLM 5.30.120.030.110.240.800.000.030.000.000.040.18MiniMax M3.10.080.030.170.141.000.000.020.010.000.050.13
Parts: every face of each custom part within 0.025 mm. Placement: each touching pair of parts on its relative pose. Ties: mates to real neighbours. Motion: joints dragged in SolidWorks. Lock-free, patterns: no Lock mates, repeats patterned. Sketches, holes, part patterns, materials: modelling practice, each weighted by the part's own geometry. Flex: rebuild after a key dimension changes.

07Findings

The best models assemble machines in opposite ways

The strongest models build assemblies in different ways. Opus 5.5 fixes most parts in space where they belong: the geometry is close, but little is mated and little moves (ties 0.07, motion 0.07, lock-free 0.21). GPT-6 Astra mates its parts and patterns repeated ones (ties 0.29, motion 0.44, lock-free 1.00, patterns 0.93) but places them less precisely (placement 0.24). Fable 5.2 does both: the best parts (0.73) and by far the best placement (0.53, against Opus 5.5's 0.28), with GPT-6 Astra-style mating (ties 0.34, motion 0.46, lock-free 1.00). Only in patterning repeated parts does it trail GPT-6 Astra (0.45 against 0.93).

Machines look right, but few parts are exact

Most machines the frontier models deliver are recognisable: 95% of GPT-6 Astra's score above 0.25. Very few parts are exact. The most common misses are angled faces and chamfers, hole sizes and positions, and lengths a fraction of a millimetre off. A part that looks right on the screen often fails face by face.

The API became the primary interface

Every model works mostly through SolidWorks' API: Python and PowerShell scripts make up 46% to 72% of all actions. Clicks and keystrokes are a small minority, used to dismiss dialogs and check the screen. How closely models read the drawings differs tenfold: median drawing views per rollout Fable 5.2 258, Opus 5.5 201, GPT-6 Astra 79, Muse Spark 26.

More time helps only a little

GPT-6 Astra takes a median 124 turns in 1.4 h; Opus 5.5 398 in 2.3 h; Fable 5.2 363 in 3.6 h (none at its longer time limit); MiniMax M3.1 1447 in 8.2 h. Longer helps only a little: within each model, reward and turns correlate weakly to moderately (Spearman ρ from -0.07 to 0.49).

08Cost and efficiency

Model$ / machine$ all 73Median turnsMedian timeTime limit hitReward
Fable 5.2$69.10$5,0453633.6 h00.573
GPT-6 Astra$47.19$3,4451241.4 h10.426
Sonnet 5.5$17.47$1,2754092.6 h00.347
Opus 5.5$24.70$1,8033982.3 h00.342
Kimi K3.1$10.77$7862883.0 h30.196
DeepSeek V4.1$1.12$823733.5 h50.127
Muse Spark$12.98$9472503.5 h50.073
GLM 5.3$3.04$2227386.7 h340.064
MiniMax M3.1$15.49$1,13114478.2 h550.024

Reward against cost

API list price per machine · mean reward
$1$3$10$300.00.10.20.30.40.50.6API cost per machine (log scale)Claude Fable 5.2Fable 5.2GPT-6 AstraGPT-6 AstraClaude Sonnet 5.5Sonnet 5.5Claude Opus 5.5Opus 5.5Kimi K3.1 ProKimi K3.1DeepSeek V4.1 FlashDeepSeek V4.1Muse Spark OpenMuse SparkGLM 5.3 FlashGLM 5.3MiniMax M3.1 FlashMiniMax M3.1
API cost from each rollout's own token counts at list prices. The SolidWorks host, a Windows machine per rollout, comes on top.

Some of the most complex machines in the set, each next to what the strongest models built from the same drawings.

Omni-Wheel Ball-Hitting Competition Robot

Volleyball robot R2 · 94 parts to model · 149 bought · 2,069 components
ReferenceFable 5.2Loading 3D…

The 3D comparison runs on a computer

ModelGeometryParts exactJoints okPlacedTimeNotes
Fable 5.20.6470/941,3352,0696.2 hPlaced 2,069 of 2,069 components where a reference part sits; 70 of 94 custom parts exactly right (mean part score 0.90); 1,335 of 5,907 joints within tolerance. Most misses: angled faces and chamfers; hole sizes and positions; worst, P227 with a face 6 mm out.
Opus 5.50.5953/949622,0694.3 hPlaced 2,069 of 2,069 components where a reference part sits; 53 of 94 custom parts exactly right (mean part score 0.87); 962 of 5,907 joints within tolerance; 1 extra components. Most misses: angled faces and chamfers; hole sizes and positions; worst, P227 with a face 27 mm out.
GPT-6 Astra0.4527/943882,0691.8 hPlaced 2,069 of 2,069 components where a reference part sits; 27 of 94 custom parts exactly right (mean part score 0.65); 388 of 5,907 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P217 with a face 17 mm out.
Sonnet 5.50.4033/9442,0693.0 hPlaced 2,069 of 2,069 components where a reference part sits; 33 of 94 custom parts exactly right (mean part score 0.69); 4 of 5,907 joints within tolerance. Most misses: flat faces (lengths, thicknesses); hole sizes and positions; worst, P201 with a face 120 mm out.
Kimi K3.10.2914/941372,0375.3 hPlaced 2,037 of 2,069 components where a reference part sits; 14 of 94 custom parts exactly right (mean part score 0.33); 137 of 5,907 joints within tolerance. Most misses: flat faces (lengths, thicknesses); hole sizes and positions; worst, P182 with a face 110 mm out.
DeepSeek V4.10.100/941472,0655.6 hPlaced 2,065 of 2,069 components where a reference part sits; 0 of 94 custom parts exactly right (mean part score 0.00); 147 of 5,907 joints within tolerance. Most misses: flat faces (lengths, thicknesses); hole sizes and positions.
GLM 5.30.084/9402,0696.7 hPlaced 2,069 of 2,069 components where a reference part sits; 4 of 94 custom parts exactly right (mean part score 0.15); 0 of 5,907 joints within tolerance. Most misses: flat faces (lengths, thicknesses); hole sizes and positions; worst, P182 with a face 139 mm out.
MiniMax M3.10.000/940112.7 hPlaced 1 of 2,069 components where a reference part sits; 0 of 94 custom parts exactly right (mean part score 0.00); 0 of 5,907 joints within tolerance.

Autonomous Quadcopter with Worm-Drive Gripper

Quadcopter with worm-drive gripper · 38 parts to model · 731 bought · 2,175 components
ReferenceFable 5.2Loading 3D…

The 3D comparison runs on a computer

ModelGeometryParts exactJoints okPlacedTimeNotes
Fable 5.20.7326/382,0402,1754.0 hPlaced 2,175 of 2,175 components where a reference part sits; 26 of 38 custom parts exactly right (mean part score 0.82); 2,040 of 5,083 joints within tolerance. Most misses: angled faces and chamfers; flat faces (lengths, thicknesses); worst, P020 with a face 8 mm out.
GPT-6 Astra0.6118/381,8042,1754.1 hPlaced 2,175 of 2,175 components where a reference part sits; 18 of 38 custom parts exactly right (mean part score 0.74); 1,804 of 5,083 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P769 with a face 13 mm out.
Sonnet 5.50.5718/381,3702,1753.9 hPlaced 2,175 of 2,175 components where a reference part sits; 18 of 38 custom parts exactly right (mean part score 0.71); 1,370 of 5,083 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P114 with a face 53 mm out.
Opus 5.50.4522/3802,1752.1 hPlaced 2,175 of 2,175 components where a reference part sits; 22 of 38 custom parts exactly right (mean part score 0.81); 0 of 5,083 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P769 with a face 39 mm out.
Kimi K3.10.2713/38102,1757.5 hPlaced 2,175 of 2,175 components where a reference part sits; 13 of 38 custom parts exactly right (mean part score 0.48); 10 of 5,083 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P027 with a face 44 mm out.
DeepSeek V4.10.160/381,0732,1477.4 hPlaced 2,147 of 2,175 components where a reference part sits; 0 of 38 custom parts exactly right (mean part score 0.00); 1,073 of 5,083 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers.
MiniMax M3.10.110/381692,17517.6 hPlaced 2,175 of 2,175 components where a reference part sits; 0 of 38 custom parts exactly right (mean part score 0.00); 169 of 5,083 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers.
GLM 5.30.00–––18.2 hRan out of time after 18.2 h without exporting the assembly STEP: nothing to grade.
Muse Spark0.00–––4.6 hThe delivered assembly STEP does not read (ValueError).

Gantry CNC Machine with Vacuum Table

CNC router · 56 parts to model · 109 bought · 1,314 components
ReferenceFable 5.2Loading 3D…

The 3D comparison runs on a computer

ModelGeometryParts exactJoints okPlacedTimeNotes
Fable 5.20.7640/566051,3144.7 hPlaced 1,314 of 1,314 components where a reference part sits; 40 of 56 custom parts exactly right (mean part score 0.91); 605 of 2,638 joints within tolerance. Most misses: angled faces and chamfers; flat faces (lengths, thicknesses); worst, P005 with a face 77 mm out.
Opus 5.50.448/5671,3142.6 hPlaced 1,314 of 1,314 components where a reference part sits; 8 of 56 custom parts exactly right (mean part score 0.74); 7 of 2,638 joints within tolerance. Most misses: angled faces and chamfers; flat faces (lengths, thicknesses); worst, P029 with a face 295 mm out.
GPT-6 Astra0.3411/56711,3141.7 hPlaced 1,314 of 1,314 components where a reference part sits; 11 of 56 custom parts exactly right (mean part score 0.52); 71 of 2,638 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P029 with a face 287 mm out.
Sonnet 5.50.286/5691,3143.6 hPlaced 1,314 of 1,314 components where a reference part sits; 6 of 56 custom parts exactly right (mean part score 0.47); 9 of 2,638 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P029 with a face 287 mm out.
Kimi K3.10.163/5601,3143.6 hPlaced 1,314 of 1,314 components where a reference part sits; 3 of 56 custom parts exactly right (mean part score 0.27); 0 of 2,638 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P013 with a face 122 mm out.
GLM 5.30.100/5601,3147.8 hPlaced 1,314 of 1,314 components where a reference part sits; 0 of 56 custom parts exactly right (mean part score 0.19); 0 of 2,638 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P080 with a face 199 mm out.
MiniMax M3.10.091/5621,27715.8 hPlaced 1,277 of 1,314 components where a reference part sits; 1 of 56 custom parts exactly right (mean part score 0.16); 2 of 2,638 joints within tolerance; 43 extra components. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P029 with a face 196 mm out.
Muse Spark0.030/5601,3143.7 hPlaced 1,314 of 1,314 components where a reference part sits; 0 of 56 custom parts exactly right (mean part score 0.05); 0 of 2,638 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P063 with a face 318 mm out.
DeepSeek V4.10.030/5601,3142.8 hPlaced 1,314 of 1,314 components where a reference part sits; 0 of 56 custom parts exactly right (mean part score 0.05); 0 of 2,638 joints within tolerance; 76 extra components. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P063 with a face 290 mm out.

Autonomous Competition Robot Main Assembly

Autonomous robot · 130 parts to model · 70 bought · 411 components
ReferenceOpus 5.5Loading 3D…

The 3D comparison runs on a computer

ModelGeometryParts exactJoints okPlacedTimeNotes
Opus 5.50.2821/130114112.4 hPlaced 411 of 411 components where a reference part sits; 21 of 130 custom parts exactly right (mean part score 0.44); 11 of 877 joints within tolerance. Most misses: angled faces and chamfers; flat faces (lengths, thicknesses); worst, P049 with a face 56 mm out.
Fable 5.20.2719/13004112.5 hPlaced 411 of 411 components where a reference part sits; 19 of 130 custom parts exactly right (mean part score 0.48); 0 of 877 joints within tolerance. Most misses: angled faces and chamfers; flat faces (lengths, thicknesses); worst, P046 with a face 52 mm out.
Sonnet 5.50.2113/13004113.8 hPlaced 411 of 411 components where a reference part sits; 13 of 130 custom parts exactly right (mean part score 0.38); 0 of 877 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P049 with a face 51 mm out.
GPT-6 Astra0.2011/13004111.6 hPlaced 411 of 411 components where a reference part sits; 11 of 130 custom parts exactly right (mean part score 0.34); 0 of 877 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P049 with a face 84 mm out.
DeepSeek V4.10.043/13004114.5 hPlaced 411 of 411 components where a reference part sits; 3 of 130 custom parts exactly right (mean part score 0.08); 0 of 877 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P166 with a face 71 mm out.
GLM 5.30.030/13044092.6 hPlaced 409 of 411 components where a reference part sits; 0 of 130 custom parts exactly right (mean part score 0.03); 4 of 877 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P169 with a face 81 mm out.
Kimi K3.10.020/13003602.7 hPlaced 360 of 411 components where a reference part sits; 0 of 130 custom parts exactly right (mean part score 0.03); 0 of 877 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P164 with a face 70 mm out.
MiniMax M3.10.00–––20.7 hRan out of time after 20.7 h without exporting the assembly STEP: nothing to grade.
Muse Spark0.00–––1.9 hThe delivered assembly STEP does not read (ValueError).

Powder-Bed Inkjet Additive Manufacturing Machine

Binder-jetting 3D printer · 70 parts to model · 67 bought · 687 components
ReferenceFable 5.2Loading 3D…

The 3D comparison runs on a computer

ModelGeometryParts exactJoints okPlacedTimeNotes
Fable 5.20.8839/701,1166875.7 hPlaced 687 of 687 components where a reference part sits; 39 of 70 custom parts exactly right (mean part score 0.90); 1,116 of 1,763 joints within tolerance. Most misses: hole sizes and positions; angled faces and chamfers; worst, P039 with a face 42 mm out.
Opus 5.50.6417/706376873.5 hPlaced 687 of 687 components where a reference part sits; 17 of 70 custom parts exactly right (mean part score 0.80); 637 of 1,763 joints within tolerance. Most misses: angled faces and chamfers; hole sizes and positions; worst, P086 with a face 175 mm out.
Sonnet 5.50.5211/701346814.1 hPlaced 681 of 687 components where a reference part sits; 11 of 70 custom parts exactly right (mean part score 0.66); 134 of 1,763 joints within tolerance. Most misses: angled faces and chamfers; hole sizes and positions; worst, P084 with a face 33 mm out.
GPT-6 Astra0.387/70816872.3 hPlaced 687 of 687 components where a reference part sits; 7 of 70 custom parts exactly right (mean part score 0.48); 81 of 1,763 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P084 with a face 49 mm out.
GLM 5.30.060/7066874.4 hPlaced 687 of 687 components where a reference part sits; 0 of 70 custom parts exactly right (mean part score 0.11); 6 of 1,763 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P086 with a face 141 mm out.
Kimi K3.10.030/7004622.5 hPlaced 462 of 687 components where a reference part sits; 0 of 70 custom parts exactly right (mean part score 0.06); 0 of 1,763 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P006 with a face 170 mm out.
DeepSeek V4.10.030/7066843.2 hPlaced 684 of 687 components where a reference part sits; 0 of 70 custom parts exactly right (mean part score 0.04); 6 of 1,763 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P060 with a face 53 mm out.

Tracked FRC Competition Robot with Shooter

FRC tracked robot with shooter · 69 parts to model · 73 bought · 564 components
ReferenceFable 5.2Loading 3D…

The 3D comparison runs on a computer

ModelGeometryParts exactJoints okPlacedTimeNotes
Fable 5.20.7737/693615644.8 hPlaced 564 of 564 components where a reference part sits; 37 of 69 custom parts exactly right (mean part score 0.78); 361 of 1,332 joints within tolerance. Most misses: angled faces and chamfers; flat faces (lengths, thicknesses); worst, P085 with a face 7 mm out.
Opus 5.50.6629/693665644.7 hPlaced 564 of 564 components where a reference part sits; 29 of 69 custom parts exactly right (mean part score 0.67); 366 of 1,332 joints within tolerance. Most misses: angled faces and chamfers; flat faces (lengths, thicknesses); worst, P112 with a face 79 mm out.
GPT-6 Astra0.3515/69475641.2 hPlaced 564 of 564 components where a reference part sits; 15 of 69 custom parts exactly right (mean part score 0.45); 47 of 1,332 joints within tolerance. Most misses: angled faces and chamfers; flat faces (lengths, thicknesses); worst, P017 with a face 67 mm out.
Sonnet 5.50.3323/69135643.3 hPlaced 564 of 564 components where a reference part sits; 23 of 69 custom parts exactly right (mean part score 0.57); 13 of 1,332 joints within tolerance. Most misses: angled faces and chamfers; flat faces (lengths, thicknesses); worst, P017 with a face 63 mm out.
Kimi K3.10.138/6905572.1 hPlaced 557 of 564 components where a reference part sits; 8 of 69 custom parts exactly right (mean part score 0.23); 0 of 1,332 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P091 with a face 105 mm out.
DeepSeek V4.10.040/6915543.4 hPlaced 554 of 564 components where a reference part sits; 0 of 69 custom parts exactly right (mean part score 0.06); 1 of 1,332 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P091 with a face 132 mm out.
Muse Spark0.032/6905624.2 hPlaced 562 of 564 components where a reference part sits; 2 of 69 custom parts exactly right (mean part score 0.05); 0 of 1,332 joints within tolerance. Most misses: flat faces (lengths, thicknesses); angled faces and chamfers; worst, P091 with a face 114 mm out.
MiniMax M3.10.00––013.3 hPlaced 0 of 564 components where a reference part sits; 0 of 69 custom parts exactly right (mean part score 0.00); 0 of 0 joints within tolerance.