Open to Opportunities Worldwide

Vamshi Krishna
Macha

AI Benchmark Engineer · Agentic AI Systems · MLOps & LLMOps · Software Engineer

✉ Hire Me View Work
5+
Years Experience
50+
Benchmark Tasks Shipped
100%
Regression Detection Rate
0.98
mAP@50 Computer Vision
About

Building the evaluation infrastructure for autonomous AI.

Software Engineer with 5 years of progressive experience architecting agentic AI evaluation systems, multi-agent benchmark frameworks, and production MLOps/LLMOps pipelines.

Deep expertise in Python, Docker, automated verification logic, and LLM-based reasoning evaluation. Built Harbor-format multi-agent task packages with reward-hacking-resistant verifiers.

Practitioner of AI coding agents (Claude Code, GitHub Copilot, Cursor) — delivering measurable outcomes at every engagement.

100%
Regression Detection RateImproved from 61.1% baseline via agentic testing
0.98
mAP@50 Computer VisionRoad monitoring across 6 international datasets
30%+
QA Overhead ReducedThrough automated verifier pipelines
40%
Manual Review ReducedWagon CV system on Indian Railways NWR
Experience

Where I've built.

Five years of progressive engineering across AI evaluation, computer vision, data engineering, and applied ML.

Jul 2026 – Present
Handshake (Turing AI Platform)
Remote · Contractor
AI Benchmark Engineer & Evaluation Specialist
  • Designed and shipped 50+ agentic AI benchmark tasks in Harbor format — multi-agent hierarchical decompositions, single-agent baselines, adversarial verifier labs, and long-horizon code synthesis challenges.
  • Authored verify.py scripts with 1:2:3 reward blend (static / reward-hacking / partial-oracle), live API round-trip spot-checks, and LLM judge integration. -30% false positives
  • Architected multi-agent DAG decomposition (dag_depth=3, dag_width=25-30) demonstrating reward 0.77-0.78 multi-agent vs 0.0 single-agent on complex knowledge tasks.
  • Built Pass@K evaluation frameworks achieving ~1e-12 NRMSE on reference validation. ~30% QA time saved
Apr 2026 – Present
AiSPRY
Hyderabad, India · Full-time
AI Research & Engineering Intern
  • Architected Wagon — a CV platform for real-time wagon inspection for Indian Railways NWR zone — using YOLOv8/YOLOv11, React, and Flask. <200ms latency -40% workload
  • Engineered defect-detection pipelines over 10,000+ inspection images, reducing manual review by ~40%.
  • Built and productionized RAG pipelines (LangChain, LlamaIndex, ChromaDB, LangSmith, Groq LLMs).
  • Operationalized MLOps/LLMOps via FastAPI, Docker, Kubernetes, and MLflow for production deployment.
Jun 2023 – Mar 2026
Self-Employed
Remote, India · Consulting
Data & AI Engineer (Independent Consulting)
  • Delivered ETL/ELT pipelines on Azure Data Factory, Databricks, and Apache Spark into Delta Lake warehouses with automated data-quality checks.
  • Built Python FastAPI microservices with MongoDB and Redis on Azure Kubernetes Service (AKS).
  • Implemented 10+ Power BI and Tableau dashboards. -35% ad-hoc requests
  • Integrated AI coding agents (GitHub Copilot, Claude Code, Cursor). ~25% faster delivery
Jan 2022 – May 2023
Freelance / Academic
India
Junior Python & ML Engineer
  • Developed CNN-based image classification models (TensorFlow, Keras) achieving >92% accuracy; published as research preprint.
  • Python/Pandas/NumPy EDA and feature engineering across structured and image datasets.
  • SQL reporting pipelines (CTEs, window functions) for business analytics use cases.
Flagship Project

Wagon.
Railways, reimagined.

Real-time computer vision inspection system for Indian Railways NWR zone — deployed at production scale with sub-200ms per-frame latency.

<200ms
Per-Frame Inference Latency
40%
Reduction in Manual Review
10K+
Image Validation Set
YOLOv8 / YOLOv11ReactFlaskDockerKubernetesMLflowEdge InferenceIndian Railways NWR
Projects

Built to measure.

Every project ships with real metrics — not vanity numbers.

⚙
SwarmBench Task Package Suite
10+ complete Harbor-format multi-agent benchmark task packages with hierarchical DAG decompositions, reward-hacking-resistant verifiers, and rubric_manifest.json scoring.
Multi-agent reward0.78 vs 0.0 single-agent
DAG depth / width3 / 25-30
Task packages10+ complete
PythonDockerYAMLHarborLLM APIs
📊
RegressionNet
End-to-end agentic regression-testing system with AI-generated tests, sandboxed execution and repair, mutation testing, skeptic analysis, and iterative orchestration. micro1 Frontier Engineering Challenge 2026.
Regression Detection Rate61.1% to 100%
Tests passing248
Regressions detected36 / 36
PythonpytestDockerMutation Testing
💡
RAG Enterprise Knowledge Assistant
Production RAG system over enterprise corpora with full LLMOps governance — prompt versioning, automated LLM evaluation, and continuous pipeline monitoring via LangSmith.
ObservabilityLangSmith end-to-end
LLM providerGroq (ultra-low latency)
Vector storeChromaDB
LangChainChromaDBGroqLangSmithFastAPI
📷
Road Infrastructure Monitoring System
Autonomous lane-marking and road-damage detection trained across 6 international datasets (USA, Canada, UK, UAE, Japan, China). Resulted in a research preprint publication.
mAP@500.98 (YOLOv11)
International datasets6 countries
OutputResearch preprint
YOLOv8YOLOv11PythonMLOps
Skills

Full-stack AI engineering.

From benchmark design to production deployment — the complete toolchain.

🎯
AI Benchmark & Eval
Harbor task format
verify.py design
rubric_manifest.json
Reward-hacking detection
Pass@K evaluation
Multi-agent DAG design
🤖
Agentic AI & LLMs
Claude Code · Copilot · Cursor
LangChain · LlamaIndex
LangSmith · Groq
RAG pipelines
Prompt engineering
LLMOps governance
🔬
Testing & QA
pytest · TDD
Mutation testing
CI/CD (GitHub Actions)
Sandboxed execution
Regression detection
Automated verifiers
🐳
Docker & Containers
Dockerfile authoring
Multi-stage builds
Docker Compose
Kubernetes / AKS
Container debugging
Reproducible envs
💾
Data Engineering
Databricks · Spark
Azure Data Factory
Apache Airflow
Delta Lake · Snowflake
ETL / ELT pipelines
Data warehousing
👁
MLOps & CV
MLflow · experiment tracking
YOLOv8 / YOLOv11
FastAPI microservices
ChromaDB · Redis
Model versioning
Drift monitoring
☁
Cloud
Azure (ADF, AKS, AI Foundry)
AWS
Oracle Cloud (OCI)
Git / GitHub
Power BI · Tableau
💻
Languages
Python (advanced)
SQL
YAML · JSON · TOML
Bash / Shell
Research

Published work.

Contributions to the research community alongside engineering practice.

Preprint · 2026
Architecting a Paradigm of Computational Intelligence for Autonomous Transportation Infrastructure Assessment
Comparative evaluation of YOLOv8 and YOLOv11 for intelligent road infrastructure monitoring across 6 international datasets · mAP@50 of 0.98
Published · 2024
Deciphering the Interplay Between Social Media Engagement, Consumer Psychology, and Purchasing Decisions
International Research Journal of Modernization in Engineering, Technology & Sciences (IRJMETS)
✉  Hire Me
Education

Academic foundation.

Multidisciplinary background spanning AI, data science, financial engineering, and business.

Diploma in Data Science & Artificial Intelligence
State University of New York (SUNY), Potsdam
Apr 2026 – Oct 2026
MS Fellowship — Financial Engineering
WorldQuant University
Oct 2025 – Nov 2027
MBA — Big Data & Business Analytics
Parul University
Jul 2022 – May 2024
BBA — Business Administration & Management
Parul University
Jul 2019 – Apr 2022
Certifications

Credentials.

Verified expertise across cloud, AI, and security domains.

Oracle Cloud Infrastructure Architect Associate
Oracle · 1Z0-1072-26
Oracle Data Science Professional
Oracle University
Oracle Generative AI Professional
Oracle University
Oracle AI Vector Search Professional
Oracle University
ISC2 CISSP
ISC2 · Aug 2026
Claude Academy: AI Capabilities & Limitations
Anthropic · Aug 2026
Claude Academy: AI Fluency — Framework & Foundations
Anthropic · Aug 2026
Claude Academy: MCP — Advanced Topics
Anthropic · Aug 2026
AI Agents Fundamentals
Hugging Face
Convolutional Neural Networks
DeepLearning.AI
Gemini Certified University Student
Google · Oct 2025 – Oct 2028
SwarmBench Trainer Certification
Turing / micro1 · 2026
Honors

Recognition.

🏆
i-Hub Innovation Challenge Grant Awardee
₹2,56,000 government innovation grant · Competitive national selection · 2021
🌎
Global Finance Trainee Honoree
Ureka Education Group, France · 2020
🥇
Gold Medal — National Science Olympiad
Science Olympiad Foundation · 2012
Contact

Ready to
work together.

Actively seeking opportunities in AI/ML Engineering, Agentic Systems, LLMOps, and Benchmark Engineering — remote or hybrid, globally.

✉  Hire Me View LinkedIn
✉
📧
📞
💼
🐙