LLM Red Team Specialist - AI Job at Mercor, New York, NY

  • Mercor
  • New York, NY

Job Description

Job Description

Job Description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: LLM Red Team Specialist — Failure Modes & Edge Cases
Type: Contract
Compensation: $60–$90/hour
Location: Remote
Commitment: 35 hours/week

Role Responsibilities

  • Evaluate frontier AI models on coding, ML , and analysis tasks to identify vulnerabilities and failure modes.
  • Design complex tasks that challenge models and are fair for grading.
  • Document findings with clear evidence and reproducible steps.
  • Collaborate with task authors to close loopholes and improve grading.
  • Share insights with researchers to enhance benchmark quality.
  • Work independently and asynchronously to meet deadlines and improve AI model performance .

Qualifications

Must-Have

  • MSc or PhD in a STEM field or equivalent experience.
  • 1+ years in research, research-engineering, security, or AI evaluation.
  • Experience identifying vulnerabilities in LLMs or ML systems .
  • Proficiency in Python and Git .
  • Familiarity with LLM capabilities and evaluation techniques.
  • Ability to work 35 hours/week .

Preferred

  • Experience in AI training , model evaluation, or benchmark/task authoring.

Resources & Support

  • For details about the interview process and platform information, please check:
  • For any help or support, reach out to: support@mercor.com

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Job Tags

Contract work, Summer work, Remote work

Similar Jobs

AHS Staffing

Travel Physical Therapy Assistant (PTA) - $1,153 per week in Oakland, CA Job at AHS Staffing

 ...Physical Therapy Assistant Location: Oakland, CA Agency: AHS Staffing Pay: $1,153 per week Shift Information: Days - 5 days x...  ...Physical Therapy Assistant in Oakland, CA for a Long Term (Travel) position. This assignment is expected to last 13 weeks... 

KPMG

Lead Specialist, Federal Data Engineer Job at KPMG

 ...Responsibilities: Leverage a variety of modern big data tools / approaches to solve complex business objectives Develop...  ...Qualifications: A minimum of five years of technical data engineering experience; U.S. Federal government consulting experience... 

Carle Health

Physical Therapist (PT) - Acute Therapy Job at Carle Health

 ...professional services and as a clinical resource for staff and students. ****Full-time and part-time positions qualifies for a sign-on bonus!!!!$7,500...  ...Health. Our nearly 17,000 team members and providers work together to support patient care across central and... 

TradeJobsWorkforce

Police Records Specialist Job at TradeJobsWorkforce

 ...As a Police Records Specialist, you will be responsible for maintaining and organizing police records, including: manage and maintain police records and databases. input data accurately and update records. respond to inquiries from officers and the public. prepare reports... 

Parrish Medical Center

Operating Room Assistant Job at Parrish Medical Center

 ...Operating Room AssistantUnder the general supervision of the Surgery Manager and the direct supervision of the Surgery Clinical Coordinator and the Operating Room RN, the Operating Room Assistant is responsible for decontamination or OR suites, restocking of supplies,...