About Turing: About TuringTuring is one of the world’s fastest-growing AI companies, accelerating the advancement and deployment of powerful AI systems. Turing helps customers in two ways: working with the world’s leading AI labs to advance frontier model capabilities in thinking, reasoning, coding, agentic behavior, multimodality, multilinguality, STEM, and frontier knowledge; and leveraging that work to build real-world AI systems that solve mission-critical priorities for companies.Role OverviewWe are looking for strong, detail-oriented software practitioners to help evaluate and improve datasets for agentic coding models.This role involves working with realistic coding tasks in an agentic coding harness, reviewing model trajectories, verifying solutions, and producing high-quality annotations.Depending on the assignment, the work may include: Online evaluations: Manually interacting with blinded models on predefined tasks, then ranking and grading resulting trajectories Offline evaluations: Designing realistic coding tasks, calibrating them through user simulation, writing task-specific rubrics, and grading generated trajectoriesThis is not a basic annotation role. Candidates are expected to read and debug code, validate behavior, follow detailed process rules, and make consistent judgment calls across model runs.We are specifically looking for candidates with enough engineering maturity to independently work on realistic software tasks, not just toy problems or shallow code-review exercises.What does day-to-day look like Execute realistic coding tasks within the assigned agentic coding harness while maintaining model blindness and session independence Follow task instructions, milestones, planned interactions, and evaluation guardrails consistently across runs Verify model outputs by reading code, running commands, checking logs, and inspecting generated artifacts Perform targeted validation of outputs using tests, scripts, and manual checks Write clear, specific, evidence-based rationales for trajectory rankings and assessments Design multi-step, realistic coding tasks (offline work), including user intent and milestone structure Create and refine task-specific rubrics and binary evaluation criteria Review completed work for quality, completeness, consistency, and schema compliance Identify and escalate broken environments, unclear instructions, or process gaps with clear supporting evidenceRequirementsSoftware Engineering Fluency (Mandatory) 5+ years of experience in software engineering, QA, developer tooling, data/ML engineering, or similar code-heavy roles Strong hands-on experience in at least 1–2 programming languages or ecosystemsRepresentative languages include :Python, JavaScript/TypeScript, Rust, Java, C/C++, Bash/CLI environments, Haskell, Swift, SQL, or other production-relevant ecosystems Ability to: Read and understand unfamiliar codebases Run and interpret tests, scripts, and CLI tools Debug issues and reason about edge cases or partial fixes Evaluate whether an implementation is functionally correctAdditional Preferred Qualifications (Offline / Senior Candidates) Strong Docker skills and experience building/debugging reproducible environments Experience working in large, complex repositories (not just small or greenfield projects) Demonstrated originality and sound engineering judgment in defining technical problems Ability to design realistic, non-trivial tasks that go beyond tutorials, README flows, or simple bug fixesPerks of Freelancing With Turing Work on cutting-edge AI projects with leading foundation model companies Collaborate on high-impact work at the frontier of LLM evaluation and reasoning Remote, flexible opportunities with global teams Competitive compensation based on experience and project scopeOffer Details Commitments Required: 8 hours per day with a 4-hour overlap with PST. Employment Type: Contractor position (Note: this role does not include medical/paid leave). Duration of Contract: 5 weeks; [expected start date is next week].
Python
JavaScript
Java
+2 more
About Turing: About TuringTuring is one of the world’s fastest-growing AI companies, accelerating the advancement and deployment of powerful AI systems. Turing helps customers in two ways: working with the world’s leading AI labs to advance frontier model capabilities in thinking, reasoning, coding, agentic behavior, multimodality, multilinguality, STEM, and frontier knowledge; and leveraging that work to build real-world AI systems that solve mission-critical priorities for companies.Role OverviewWe are looking for strong, detail-oriented software practitioners to help evaluate and improve datasets for agentic coding models.This role involves working with realistic coding tasks in an agentic coding harness, reviewing model trajectories, verifying solutions, and producing high-quality annotations.Depending on the assignment, the work may include: Online evaluations: Interacting with blinded models on predefined tasks, then ranking and grading the resulting trajectories Offline evaluations: Designing realistic coding tasks, calibrating them through user simulation, writing task-specific rubrics, and grading generated trajectoriesThis is not a basic annotation role. It requires strong engineering judgment, the ability to read and debug code, validate behavior, follow detailed process rules, and make consistent evaluation decisions across model runs.We are specifically looking for candidates with enough engineering maturity to independently handle real-world software tasks, not just toy problems or superficial code review exercises.What does day-to-day look like Execute realistic coding tasks within the assigned agentic coding harness while maintaining model blindness and session independence Follow task instructions, milestones, planned interactions, and evaluation guardrails consistently across runs Verify model outputs by reading code, running commands, checking logs, and validating generated artifacts Write clear, specific, evidence-based rationales for trajectory rankings and assessments Design multi-step, realistic coding tasks (offline work), including user intent, milestones, and expected behaviors Create and refine task-specific rubrics and binary evaluation criteria Review completed work for quality, consistency, completeness, and schema compliance Identify and escalate broken environments, unclear instructions, or process gaps with supporting evidenceRequirementsSoftware Engineering Fluency (Mandatory) 5+ years of experience in software engineering, QA, developer tooling, data/ML engineering, or similar code-heavy roles Strong hands-on experience in at least 1–2 programming languages or ecosystems Ability to:Read and understand unfamiliar codebasesRun and interpret tests, scripts, and CLI toolsDebug issues and reason about edge cases or partial fixesEvaluate whether an implementation is functionally correctRepresentative languages include:Python, JavaScript/TypeScript, Rust, Java, C/C++, Bash/CLI environments, Haskell, Swift, SQL, or other production-relevant ecosystemsTerminal & Tooling Skills (Mandatory) Comfortable working in Linux/Ubuntu-like environments Proficient with:Terminal workflowsGit basicsCode editors or IDEsPackage managers and test runnersJSON, YAML, and Markdown Familiarity with Docker and reproducible environments (strong plus, especially for offline work)Coding-Agent Workflow Familiarity (Mandatory) Comfortable working with or quickly adapting to agentic coding environments, such as:OpenCodeClaude CodeCursorSimilar coding-agent tools Ability to execute and evaluate tasks within agent-driven workflowsQuality Judgment & Annotation Accuracy (Mandatory) Ability to:Compare multiple model trajectories and identify meaningful differencesDistinguish correctness from style, communication quality, and agent behaviorEvaluate solutions consistently using defined rubricsFollow detailed process instructions without deviationMaintain consistency across repeated or similar evaluationsWrite concise, evidence-based rationales (not generic summaries)Work Style Highly detail-oriented and process-driven Comfortable with repetitive, high-precision evaluation work Able to maintain consistency across long tasks and multiple model runs Proactively flags ambiguity instead of making assumptions Balances realism with strict evaluation consistencyPerks of Freelancing With Turing Work on cutting-edge AI projects with leading foundation model companies Collaborate on high-impact work at the frontier of LLM evaluation and reasoning Remote, flexible opportunities with global teams Competitive compensation based on experience and project scopeOffer Details Commitments Required: 8 hours per day with a 4-hour overlap with PST. Employment Type: Contractor position (Note: this role does not include medical/paid leave). Duration of Contract: 5 weeks; [expected start date is next week].
Python
JavaScript
Java
+2 more
About Turing: About Turing:Based in San Francisco, California, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing supports customers in two ways: first, by accelerating frontier research with high-quality data, advanced training pipelines, plus top AI researchers who specialize in coding, reasoning, STEM, multilinguality, multimodality, and agents; and second, by applying that expertise to help enterprises transform AI from proof of concept into proprietary intelligence with systems that perform reliably, deliver measurable impact, and drive lasting results on the P&LJob OverviewWe are seeking professionals with strong scientific reasoning and quantitative analysis skills to create high-quality reasoning datasets for Large Language Model training. This role focuses on designing scientific discovery tasks in which models analyze experimental or simulated data, identify patterns, infer formulas or laws, and apply them to new cases. Ideal candidates have a solid understanding of the scientific method, experimental design, mathematical modeling, and data interpretation across fields such as physics, chemistry, and biology. They should be able to design clear, challenging tasks that test whether a model can move from observation to explanation in a precise and verifiable way.Key Responsibilities Design scientific scenarios using experimental data, observational records, or simulated systems, including fictional but internally consistent systems. Author multi-step reasoning tasks that require models to analyze data, identify relationships, infer governing rules, estimate parameters, and predict outcomes. Develop problem types such as law induction from data, parameter recovery, model selection, predictive reasoning, and impossible-scenario detection. Create clear context blocks, prompts, expected answers or reference solutions, and scoring rubrics for each task. Anticipate edge cases such as noisy data, boundary conditions, impossible states, missing assumptions, and unit inconsistencies. Collaborate with reviewers and LLM engineers to improve task clarity, scientific accuracy, and reproducibility. Maintain high quality standards across all tasks, with strong attention to logic, precision, and internal consistency.Qualifications 3+ years of experience in a scientific, research, or analytical role such as physics, chemistry, scientific computing, or data analysis. Strong background in scientific reasoning, quantitative modeling, and data interpretation. Familiarity with experimental design, mathematical modeling, parameter estimation, dimensional analysis, or simulation-based reasoning. Proven ability to design structured reasoning problems and explain them clearly in writing. Strong attention to detail, especially around assumptions, ambiguity control, and consistency between data, solution, and rubric. Experience working with or evaluating Large Language Models is a plus. A degree in Physics, Chemistry, Biology, or a related science discipline is preferred.Deliverables A structured collection of reasoning tasks covering law discovery from data, inference from observational or simulated systems, parameter estimation, prediction under new conditions, and scientific consistency checks. Each task should include: A formal context block with assumptions, variables, definitions, and observations A prompt asking the model to infer, compute, justify, predict, or explain An expected output with a deterministic final answer and reference solution A rubric for evaluating correctness, reasoning quality, and completeness Final tasks must be self-contained, reproducible, and aligned with model reasoning evaluation goals. Offer Details: Commitments Required: at least 4 hours per day and minimum 40 hours per week with 4 hours of overlap with PST. Engagement type: Contractor Engagement Length: 5 weeksEvaluation Process - Shortlisted candidates will be sent a Job Interest Form. Shortlisted candidates will be contacted to discuss the pre‑onboarding requirements
About Turing: About Turing:Based in San Francisco, California, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing supports customers in two ways: first, by accelerating frontier research with high-quality data, advanced training pipelines, plus top AI researchers who specialize in coding, reasoning, STEM, multilinguality, multimodality, and agents; and second, by applying that expertise to help enterprises transform AI from proof of concept into proprietary intelligence with systems that perform reliably, deliver measurable impact, and drive lasting results on the P&L.Role Overview:We are looking for candidates with strong analytical and English/Italian comprehension skills. The ideal candidate should have the ability to read, summarize, and break down large content into smaller logical blocks, conduct research online, validate claims made in the content through online research, and work with the LLM (Large Language Models) to solve puzzles!Your role is critical in helping fine-tune and improve large language models (like gpt), and will make you an expert on how to leverage AI to be a better analyst. This is your chance to future-proof your career in an AI-first world!What does day-to-day look like: Listen to audio recordings and accurately transcribe spoken content following the provided syntax and formatting guidelines. Annotate non-verbal cues (e.g., pauses, laughter, tone changes, background reactions) to capture the full context of each recording. Review and edit transcriptions to ensure they are contextually accurate, complete, and compliant with project-specific standards. Identify and report inconsistencies or ambiguities in speech or annotation, providing clear feedback for quality improvement. Deliver high-quality audio files on time and in the correct format Collaborate with cross-functional teams or follow detailed instructions to maintain high-quality standards in transcription and annotation accuracy.Requirements: Multilingual capabilities are mandatory (English and Italian). Analytical Skills: Good research and analytical skills Feedback Skills: Ability to provide constructive feedback and detailed annotations. Creative Thinking: Creative and lateral thinking abilities. Communication: Excellent communication and collaboration skills. Independence: Self-motivated and able to work independently in a remote setting. Commitment: Ability to commit to 40 hours per week for the contract duration overlapping US hours. Technical Setup: headphones and a working microphone.Preferred Qualifications: Bachelors degree or undergraduate in Engineering, Literature, Journalism, Communications, Arts, Statistics, or a related field. We are open to candidates who do not have a Bachelor's degree but have experience in the area. Experience with audio related RLHF tasks and/or voice acting is highly desirable. Experience writing professionally (business analysts, research analyst, copywriter, journalist, technical writer, editor, translator, etc.) Understanding of Excel and Google Suite. Proficiency in Data interpretation, Logical reasoning and Basic arithmetic is highly desirable.Perks of Freelancing With Turing: Competitive compensation based on experience and expertise. Flexible working hours and remote work environment. Opportunity to work on cutting-edge AI projects with leading LLM companies. Potential for contract extension based on performance and project needs.Offer Details: Commitments Required: Availability of upto 40 hours per week is preferred Engagement type: Contractor assignment/freelancer (no medical/paid leave) Duration of contract 10 Weeks inital pilot, could extend. This role will require 4 hour overlap with PT Timezone.Application Process: Shortlisted candidates will be sent an automated analytical challenge. Once you clear the challenge, you will be redirected to a business writing assessment (English) followed by a language proficiency assessment. Once you clear these online assessments, you are ready to go!
About Turing: About Turing:Based in San Francisco, California, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing supports customers in two ways: first, by accelerating frontier research with high-quality data, advanced training pipelines, plus top AI researchers who specialize in coding, reasoning, STEM, multilinguality, multimodality, and agents; and second, by applying that expertise to help enterprises transform AI from proof of concept into proprietary intelligence with systems that perform reliably, deliver measurable impact, and drive lasting results on the P&L.Role Overview:We are looking for candidates with strong analytical and English/German comprehension skills. The ideal candidate should have the ability to read, summarize, and break down large content into smaller logical blocks, conduct research online, validate claims made in the content through online research, and work with the LLM (Large Language Models) to solve puzzles!Your role is critical in helping fine-tune and improve large language models (like gpt), and will make you an expert on how to leverage AI to be a better analyst. This is your chance to future-proof your career in an AI-first world!What does day-to-day look like: Listen to audio recordings and accurately transcribe spoken content following the provided syntax and formatting guidelines. Annotate non-verbal cues (e.g., pauses, laughter, tone changes, background reactions) to capture the full context of each recording. Review and edit transcriptions to ensure they are contextually accurate, complete, and compliant with project-specific standards. Identify and report inconsistencies or ambiguities in speech or annotation, providing clear feedback for quality improvement. Deliver high-quality audio files on time and in the correct format Collaborate with cross-functional teams or follow detailed instructions to maintain high-quality standards in transcription and annotation accuracy.Requirements: Multilingual capabilities are mandatory (English and German). Analytical Skills: Good research and analytical skills Feedback Skills: Ability to provide constructive feedback and detailed annotations. Creative Thinking: Creative and lateral thinking abilities. Communication: Excellent communication and collaboration skills. Independence: Self-motivated and able to work independently in a remote setting. Commitment: Ability to commit to 40 hours per week for the contract duration overlapping US hours. Technical Setup: headphones and a working microphone.Preferred Qualifications: Bachelors degree or undergraduate in Engineering, Literature, Journalism, Communications, Arts, Statistics, or a related field. We are open to candidates who do not have a Bachelor's degree but have experience in the area. Experience with audio related RLHF tasks and/or voice acting is highly desirable. Experience writing professionally (business analysts, research analyst, copywriter, journalist, technical writer, editor, translator, etc.) Understanding of Excel and Google Suite. Proficiency in Data interpretation, Logical reasoning and Basic arithmetic is highly desirable.Perks of Freelancing With Turing: Competitive compensation based on experience and expertise. Flexible working hours and remote work environment. Opportunity to work on cutting-edge AI projects with leading LLM companies. Potential for contract extension based on performance and project needs.Offer Details: Commitments Required: Availability of upto 40 hours per week is preferred Engagement type: Contractor assignment/freelancer (no medical/paid leave) Duration of contract 10 Weeks inital pilot, could extend. This role will require 4 hour overlap with PT Timezone.Application Process: Shortlisted candidates will be sent an automated analytical challenge. Once you clear the challenge, you will be redirected to a business writing assessment (English) followed by a language proficiency assessment. Once you clear these online assessments, you are ready to go!