Rakesh Chowdary Machineni

I fine-tune and deploy vision-language-action policies on real robots, and build the real-to-sim-to-real pipeline that trains them.

Email LinkedIn Google Scholar CV (PDF)

I am a Research Scientist on the Artificial Intelligence Center of Excellence (AI-CoE) team at Rockwell Automation in Austin, TX, where I lead the lab's robot manipulation effort.

My current work is on making off-the-shelf vision-language-action (VLA) models usable on industrial tasks: a framework for profiling policies such as MolmoAct2, SmolVLA, and π0 on a benchmark of manipulation tasks, visual prompting to fix them where language grounding fails, and a real-to-sim-to-real pipeline in NVIDIA Isaac Sim that generates training data without teleoperation. On mobile robots I have deployed OmniVLA, LightNav-0, and a sim-trained NoMaD policy on a production OTTO 100 AMR.

Before robot learning I built LenzAI, a closed-loop industrial machine-vision platform running at 3+ customers, and showed that frozen V-JEPA2 video features work as a label-free sensor and control signal for industrial processes. I am a named inventor on eight pending U.S. patent applications.

I received my M.S. in Electrical and Computer Engineering (Computer Vision) from the University of Michigan, advised by Prof. Hun-Seok Kim, where I worked on learned video compression (CVPR 2023), and my B.Tech. in Electrical Engineering from IIT Tirupati.

Rakesh Chowdary Machineni

Research

VLA policies for industrial manipulation · Jan 2026 – present

A framework for profiling and deploying VLA policies, and visual prompting where they fail.

  • Designed a framework and a benchmark of industrial manipulation tasks for evaluating off-the-shelf VLA policies on an SO-100 arm. The pipeline fine-tunes a policy where reliability is low and tests its language-grounding and reasoning capabilities when it fails. Applied to SmolVLA, π0, and MolmoAct2.
  • Identified the failure modes of VLA policies at two model scales as language grounding and reasoning rather than control. Small VLAs could not tell which object an instruction referred to (red vs. blue cube), and MolmoAct2, which could, still failed instructions involving spatial relations or multiple steps: 1/10 (nearest/farthest pick-up), 0/10 (left/right pick-up), 0/12 ("clear the table").
  • Developed visual prompting as the framework's remedy when the skill is present but grounding or reasoning is absent. An external VLM decomposes the instruction and marks target and destination on the policy's input image; a fine-tune on the same data with overlays raised the three tasks to 10/10, 9/10, and 11/12.
  • Built a real-to-sim-to-real pipeline that reconstructs the workbench in Isaac Sim and generates training episodes without teleoperation; fine-tuning MolmoAct2 on them improved real-arm success over the zero-shot checkpoint with sim-only data and further with sim and teleoperated real data combined.

Navigation foundation models on a production AMR · Nov – Dec 2025

Open-weight navigation models evaluated, trained in simulation, and deployed on an OTTO 100.

  • Deployed OmniVLA and LightNav-0 on a production, safety-rated OTTO 100 AMR via a custom ROS/Docker bridge; evaluated language, goal-image, and goal-pose modalities on object-directed instructions.
  • Trained NoMaD, a goal-free exploration policy, in Isaac Sim on an embodiment matched to the OTTO 100 and transferred it to the real robot for autonomous exploration.

Closed-loop machine vision from video streams (V-JEPA2) · Jul – Nov 2025

Frozen video world-model features as a label-free sensor and control signal for industrial processes.

  • Latents from a frozen V-JEPA2 encoder cluster by operating state with no training: a safety signal for unsafe forklift handling (forks mispositioned or dropped on the load), and coarse vs. fine rock classification on a conveyor at 96% accuracy.
  • Used the same clusters as a control signal for an AMR in Isaac Sim, driving its speed from 1.5 to 0.5 m/s from video alone (U.S. Pat. App. 19/819,907).
  • Adapted VTCD concept discovery (SLIC tubelets + CNMF) with a DINOv2 object-of-interest gate to track a chosen object from a single reference crop, with no detector or labels (U.S. Pat. App. 19/822,786).

LenzAI: industrial machine vision in production · Jul 2023 – Jul 2025

A closed-loop machine-vision platform for factory operators, and the inspection models that run on it.

  • Labeling, training, monitoring, and analytics in one loop, deployed across 3+ paying customers with 99% uptime; integrated with Rockwell and Siemens PLCs and Cognex/Keyence/GenICam cameras at sub-50 ms latency.
  • Anomaly detection on a casting line with a normalizing-flow model trained only on normal operation: flagged all 5 die-stick and short-pour events across 60,000+ images over 18 months, with no labeled defects.
  • Unsupervised depth-based wrinkle detection for automotive seat inspection (83% precision / 97% recall); Molmo + SAM2 point-prompt labeling (10× faster annotation); model-health monitoring that detects data shift and triggers active learning.

News

November 2026

Attending CoRL 2026 in Austin.

October 2026

Five U.S. patent applications filed, including two on vision-language-action models and one on vision-based soft sensing for closed-loop control.

September 2026

"LLM-MLFFN: Multi-Level Autonomous Driving Behavior Feature Fusion via Large Language Model" presented as an oral at IEEE ITSC 2026, Naples.

September 2026

Two U.S. patent applications filed on monitoring industrial operations from multi-modal sensors and on synthetic anomaly data.

July 2026

EMAGN preprint on scalable traffic forecasting released on arXiv.

March 2026

LLM-MLFFN preprint released on arXiv.

June 2025

U.S. patent application filed: "Systems and Methods for Identifying and Controlling Operational Targets."

June 2025

SheetMind preprint released on arXiv.

July 2023

Started as a Research Scientist at Rockwell Automation, Austin, TX.

May 2023

Graduated from the University of Michigan with an M.S. in ECE (Computer Vision).

February 2023

"MMVC: Learned Multi-Mode Video Compression with Block-based Prediction Mode Selection and Density-Adaptive Entropy Coding" accepted to CVPR 2023.

January 2022

Joined Prof. Hun-Seok Kim's lab as a Graduate Research Assistant.

August 2021

Began M.S. at the University of Michigan, Ann Arbor.

July 2019

Joined Toshiba Software as a Computer Vision Engineer.

May 2019

"End-to-end deep learning-based fringe projection framework for 3D profiling of objects" accepted to Computer Vision and Image Understanding.

Contact

Email rakeshc.machineni@gmail.com or connect on LinkedIn.