About us

At JetBrains, code is our passion. Ever since we started, we have strived to make the strongest, most effective developer tools on earth. By automating routine checks and corrections, our tools speed up production, freeing developers to grow, discover, and create.

We believe in building tools developers love - and we see the next frontier in AI-powered developer tools. The DPAI Arena project aims to define and maintain an open, community-driven benchmark for evaluating AI agents and IDE-embedded AI features at scale. We believe that the benchmark industry is only in its early stages, and we're planning to take part in the evolution of the field. 

We're looking for a Senior Product Manager to own the direction of DPAI Arena and help define what "good evaluation" looks like and how we measure it. 

What you'll do

• Refine our answer to the question of what makes an evaluation task a representative data point for software engineers and IDE AI features, using usage data, feedback loops, and community input.
• Define and iterate the evaluation methodology, designing tasks, specifying metrics and success criteria, and covering real developer workflows from both product and UX perspectives.
• Build and maintain relationships with external vendors, platform partners, outsourcing providers, and internal stakeholders to expand participation in DPAI Arena and drive both business and ecosystem development.
• Curate and manage datasets, coordinating contributions from JetBrains teams, external partners, and outsourcing vendors (including crowdsourced workers when needed) to keep evaluation tasks high-quality, representative, and diverse.
• Own the evaluation standard and roadmap, representing DPAI Arena internally and externally while aligning plans with the broader JetBrains AI and IDE strategy.
  
  

What we expect from you

• Proven experience in product management (developer tools or data-driven products is a plus).
• Experience working on GenAI features for developer tooling.
• Strong analytical skills, with the ability to define meaningful metrics, interpret results, and turn them into product design.
• Experience working with multiple stakeholders: internal teams, external partners, and vendors (including outsourcing providers).
• Understanding of developer workflows, IDEs, and AI-assisted development, and the ability to translate real-world coding tasks into evaluation workloads.
• Strong communication skills, both written and spoken.
• Ownership mindset and proactive approach.
• Engineering background (particularly SWE or QA for tech products).
  
  

Bonus points

• Experience maintaining or curating open-source products.
• ML experience. 
• Familiarity with outsourcing or crowdsourcing platforms, data collection pipelines, or community-driven ecosystems.
  

  Why join JetBrains?

  • Strong base salary. We offer competitive pay that reflects your skills and experience.
  • Flexible work location. Enjoy the freedom to work from home or from the office.
  • Remote work. Spend up to 30 days per year working remotely from abroad.
  • Extra time off. More days to relax, recharge, and do the things you love.
  • Medical insurance allowance. Enjoy peace of mind for you and your family.
  • Learning and development opportunities. Access to conferences, courses, and language classes.
  • Relocation support. We help make your move as smooth and stress-free as possible.
  • Language classes. Pick up the local language or sharpen your English skills.
  • Fuel your day. Enjoy a hot meal or receive a lunch allowance on workdays.
  • Mental health support. To help you feel your best, we provide easy access to professional mental health services.
  • Sports benefit. Enjoy an on-site gym or sports club stipend.
  • Internal events. Join company-wide celebrations and team gatherings.

  *Some benefits may vary depending on location.

  
  
  

#LI-HYBRID 

#LI-AA1