Physical Adversarial Attacks vs The Sim-to-Real Gap

A researcher at Two Six Technologies shares two failed approaches to creating physical adversarial attacks against computer vision object detectors using everyday materials. The first attempt used colored duct tape squares mapped to body keypoints, optimized in simulation to hide a person from detection. The second used rigid cardboard and face masks covered in tape. Both failed due to the sim-to-real gap: simulations couldn't account for how materials conform to body contours, cast shadows, and shift angles in the real world. The post serves as a candid postmortem on dead ends before a forthcoming paper reveals the successful approach.