Staff Software Engineer, LLMs

Upstart is seeking a Staff Software Engineer to develop foundational infrastructure for generative AI across the company's product teams through the Applied LLM team.

Key Responsibilities

• Design and build user-facing ML features that harness LLMs and generative AI
• Partner with product, design, and ML research to prototype intelligent experiences
• Own technical architecture for applied ML systems, balancing performance considerations
• Build scalable services and APIs for model outputs
• Collaborate across platform, infrastructure, and compliance teams
• Establish best practices for prompt design and model evaluation

Required Qualifications

• 6+ years software engineering experience with 2+ years on ML-driven products
• Proven leadership across engineering, product, and research stakeholders
• Strong backend development (Python, FastAPI/Flask)
• Cloud-native tooling expertise (Kubernetes, Docker, Terraform)
• Production LLM/ML model integration experience
• Excellent communication and collaborative approach

Preferred Qualifications

• GenAI/LLM framework experience (LangChain, LlamaIndex, OpenAI APIs)
• RAG, vector search, and real-time inference knowledge
• Full-stack development capabilities
• Prompt engineering and model evaluation expertise
• Compliance and explainability experience
• Track record influencing architecture at team/org level

Compensation & Benefits
US Remote: $199,500-$276,000 USD base
Additional benefits include bonuses, equity, 401(k) matching ($2:$1 up to $15,000/year), medical/dental/vision (90-100% covered), parental leave, wellness allowance, productivity allowance.

Location & Travel
Remote (US/Canada), operating on East/West Coast time zones. Quarterly on-site collaboration (3-4 days per quarter); Upstart covers travel expenses.