Job description
Software Engineer - HPC/LLMs - 7+ yrs of Experience - TS/SCI w/Poly clearance is required Halogen Engineering is seeking a Software Engineer who will design, develop, deploy, and support Retrieval-Augmented Generation (RAG) solutions within a high-performance computing (HPC) Linux environment. The successful candidate will have hands-on experience with Large Language Models (LLMs), LLM orchestration frameworks, knowledge retrieval systems, embedding models, and containerized applications. Candidates must be proficient in Python, Kubernetes, Helm, CI/CD automation, and secure SI system development, with the ability to deliver scalable, production-ready SI solutions in mission-critical environments. This candidate will design, develop, test, deploy, document, maintain, and enhance complex and diverse software systems based upon documented requirements. These systems might include, but are not limited to, processing intensive analytics, novel algorithm development, manipulation of extremely large data sets, real-time systems, business management information systems, and systems which incorporate data repositories, data transport services, and application and systems development and monitoring. Works individually or as part of a team. Reviews and tests software components for adherence to the design requirements and documents test results. Resolves software problem reports. Utilizes software development and software design methodologies appropriate to the development environment. Provides specific input to the software components of system design to include hardware/software trade-offs, software reuse, use of Open Source Software (OSS) and/or Commercial OffThe-Shelf (COTS) Government Off-The-Shelf (GOTS) software in place of new development, and requirements analysis and synthesis from system level to individual software components. Experience developing in Unix. Ability to perform shell scripting. Working knowledge of Configuration Management (CM) tools and Web Services implementation. Location: Annapolis Junction, MD Travel Required: No Shift: Day Remote Type: Onsite Security Clearance: Current TS/SCI Polygraph: Required (must be less than seven years old) Salary Range The projected compensation range for this position is 227K to 241K (annualized USD). Salary is determined by various factors, including but not limited to location, the individual’s particular combination of education,knowledge, skills, competencies, and experience, as well as contract-specific affordability and organizational requirements. Capabilities Analyze user requirements to derive software design and performance requirements. Debug existing software and correct defects. Design and code new software or modify existing software to add new features. Write or review software and system documentation. Integrate existing software into new or modified systems or operating environments. Develop simple data queries for existing or proposed databases or data repositories. Software development using languages such as C, C++, Python, Ruby, Perl, JavaScript, etc. Has experience with agile development processes. Has experience with source code control systems, such as Git. Serve as team lead at the level appropriate to the software development process being used on any particular project. Design and development of relational and non-relational database applications. Use of orchestration frameworks such as Spring and Kafka. Familiarization with queue management systems Develop or implement algorithms to meet or exceed system performance and functional standards. Develop and execute test procedures for software components. Develop software solutions by analyzing system performance standards and conferring with users or system engineers; analyzing systems flow, data usage and work processes; and investigating problem areas. Modify existing software to adapt to new hardware or to improve its performance. Design, develop, and modify software systems using scientific analysis and mathematical models to predict and measure outcomes and consequences of design decisions. Java development using the Eclipse IDE (Integrated Development Environment). Development of Java 2 Enterprise Edition (J2EE) applications Experience using collaboration and software development tools (ie. Atlassian). Software development using continuous integration practices. Experience with container technologies (ie. Docker). Unix shell scripting Development of event driven, or data driven analytics Development of cloud-based solutions and technologies. Design or implement complex algorithms requiring adherence to strict timing, system resource, or interface constraints. Perform quality control on team products. Recommend and implement suggestions for improving documentation and software development process standards. Oversee one or more software development teams and ensure the work is completed in accordance with the constraints of the software development process being used on any particular project. Confer with system engineers and hardware engineers to derive software requirements and to obtain information on project limitations and capabilities, performance requirements, and interfaces. Coordinate software installation on a system and monitor performance to ensure operational specifications are met. Recommend new technologies and processes for complex software projects. Serve as the technical lead of multiple software development teams. Select the software development process in coordination with the customer and system engineering. Ensure quality control of all developed and modified software. Delegate programming and testing responsibilities to one or more teams and monitor their performance. Requirements Required Skills Proven experience working in Linux environments, including advanced use of the Linux command-line interface (CLI) for development, troubleshooting, and system administration Recent hands-on software development experience using Python, with a strong understanding of modern software engineering practices and frameworks Experience with containerization technologies, such as Docker, Podman, and containerd, for application packaging, deployment, and lifecycle management Experience designing, authoring, and deploying Helm charts to manage Kubernetes-based applications and services Experience developing, testing, deploying, and supporting containerized applications in production environments. Experience creating, configuring, and managing Kubernetes resources, including Deployments, Services, ConfigMaps, Secrets, Ingresses, and Operators Working knowledge of Generative SI technologies, including Large Language Models (LLMs), Retrieval-Augmented Generation (RAG) architectures, embedding models, and vector databases Experience implementing and maintaining CI/CD pipelines using tools such as GitLab CI/CD, Flux CD, GitLab Runners or similar tools Experience leveraging observability, metrics, logging, and monitoring solutions including Prometheus and Grafana Proficiency with Git-based source control workflows, including branching strategies, code reviews, pull/merge requests, and collaborative development practices Understanding of software development lifecycle (SDLC) best practices, including automated testing, security scanning, code quality analysis, and DevSecOps principles Desired Skills Familiar with developing, deploying, troubleshooting, and sustaining mission applications in an AWS Cloud environment Experience with the Atlassian suite of tools including Confluence and Jira Experience working with, and debugging, GPU-enabled applications Experience using a machine-learning framework (PyTorch, TensorFlow, etc) Experience with other ML pipelines/frameworks like KubeFlow, NeMo, PyTorch Lightning, etc. Experience working with and debugging GPU-enabled applications Experience with SQL, Elasticsearch, and Vector databases Knowledge of SI inferencing solutions such as Nvidia NIM/TRITON, vLLM, and direct deployment (e.g. Ray) Qualifications Master’s degree in computer science or related discipline from an accredited college or university, plus five (5) years of experience as a SWE, in programs and contracts of similar scope, type, and complexity. OR Bachelor’s degree in computer science or related discipline from an accredited college or university, plus seven (7) years of experience as a SWE, in programs and contracts of similar scope, type, and complexity OR Nine (9) years of experience as a SWE, in programs and contracts of similar scope, type, and complexity. Benefits CareFirst Insurance Medical Plans Three available medical plans–2 Platinum PPO Plans ($0 Deductible) and 1 HSA Plan ($1500 deductible) Halogen pays 100% Medical Premium Costs up to $25,000 CareFirst Wellness Rewards Program Guardian Insurance Dental Plan Halogen pays 50% of the plan costs Vision Plan Halogen pays 100% of the plan costs 401K Retirement Plan · 10% contribution paid by the company, automatically vested and paid in each pay period. Life Insurance, AD&D, Short and Long Term Disability Coverage Paid Time Off · 20 Paid Time Off Days · 6 Federal Holidays · 5 Floating Holidays Bonuses · Based on performance and billable hours Halogen Engineering Group does not discriminate in employment on the basis of race, color, religion, sex (including pregnancy and gender identity), national origin, political affiliation, sexual orientation, marital status, disability, genetic information, age, membership in an employee organization, retaliation, parental status, military service, or other non-merit factor.