Staff Software Engineer, LLMs at Upstart

Upstart is seeking a Staff Software Engineer to develop foundational infrastructure for generative AI across the company's product teams through the Applied LLM team.

Key Responsibilities

Design and build user-facing ML features that harness LLMs and generative AI

Partner with product, design, and ML research to prototype intelligent experiences

Own technical architecture for applied ML systems, balancing performance considerations

Build scalable services and APIs for model outputs

Collaborate across platform, infrastructure, and compliance teams

Establish best practices for prompt design and model evaluation

Required Qualifications

6+ years software engineering experience with 2+ years on ML-driven products

Proven leadership across engineering, product, and research stakeholders

Strong backend development (Python, FastAPI/Flask)

Cloud-native tooling expertise (Kubernetes, Docker, Terraform)

Production LLM/ML model integration experience

Excellent communication and collaborative approach

Preferred Qualifications

GenAI/LLM framework experience (LangChain, LlamaIndex, OpenAI APIs)

RAG, vector search, and real-time inference knowledge

Full-stack development capabilities

Prompt engineering and model evaluation expertise

Compliance and explainability experience

Track record influencing architecture at team/org level

Compensation & Benefits
US Remote: $199,500-$276,000 USD base
Additional benefits include bonuses, equity, 401(k) matching ($2:$1 up to $15,000/year), medical/dental/vision (90-100% covered), parental leave, wellness allowance, productivity allowance.

Location & Travel
Remote (US/Canada), operating on East/West Coast time zones. Quarterly on-site collaboration (3-4 days per quarter); Upstart covers travel expenses.