Job description
Software Engineer - Apache Spark/Big Data - 7+ yrs of Experience - TS/SCI w/Poly clearance is required Halogen Engineering Group is looking for a Software Engineer who will possess extensive expertise in dataflow design, data transport mechanisms, and Apache Spark based distributed processing. In this role, the Software Engineer shall be responsible for designing, implementing, and optimizing data ingress/egress pathways to ensure efficient, scalable, and reliable processing of the organization’s analytics workloads. This candidate will design, develop, test, deploy, document, maintain, and enhance complex and diverse software systems based upon documented requirements. These systems might include, but are not limited to, processing intensive analytics, novel algorithm development, manipulation of extremely large data sets, real-time systems, business management information systems, and systems which incorporate data repositories, data transport services, and application and systems development and monitoring. Works individually or as part of a team. Reviews and tests software components for adherence to the design requirements and documents test results. Resolves software problem reports. Utilizes software development and software design methodologies appropriate to the development environment. Provides specific input to the software components of system design to include hardware/software trade-offs, software reuse, use of Open Source Software (OSS) and/or Commercial OffThe-Shelf (COTS) Government Off-The-Shelf (GOTS) software in place of new development, and requirements analysis and synthesis from system level to individual software components. Experience developing in Unix. Ability to perform shell scripting. Working knowledge of Configuration Management (CM) tools and Web Services implementation. Location: Fort Meade, MD Travel Required: No Shift: Day Remote Type: Onsite Security Clearance: Current TS/SCI Polygraph: Required (must be less than seven years old) Salary Range The projected compensation range for this position is 227K-241K (annualized USD). Salary is determined by various factors, including but not limited to location, the individual’s particular combination of education,knowledge, skills, competencies, and experience, as well as contract-specific affordability and organizational requirements. Capabilities Analyze user requirements to derive software design and performance requirements. Debug existing software and correct defects. Design and code new software or modify existing software to add new features. Write or review software and system documentation. Integrate existing software into new or modified systems or operating environments. Develop simple data queries for existing or proposed databases or data repositories. Software development using languages such as C, C++, Python, Ruby, Perl, JavaScript, etc. Has experience with agile development processes. Has experience with source code control systems, such as Git. Serve as team lead at the level appropriate to the software development process being used on any particular project. Design and development of relational and non-relational database applications. Use of orchestration frameworks such as Spring and Kafka. Familiarization with queue management systems Develop or implement algorithms to meet or exceed system performance and functional standards. Develop and execute test procedures for software components. Develop software solutions by analyzing system performance standards and conferring with users or system engineers; analyzing systems flow, data usage and work processes; and investigating problem areas. Modify existing software to adapt to new hardware or to improve its performance. Design, develop, and modify software systems using scientific analysis and mathematical models to predict and measure outcomes and consequences of design decisions. Java development using the Eclipse IDE (Integrated Development Environment). Development of Java 2 Enterprise Edition (J2EE) applications Experience using collaboration and software development tools (ie. Atlassian). Software development using continuous integration practices. Experience with container technologies (ie. Docker). Unix shell scripting Development of event driven, or data driven analytics Development of cloud-based solutions and technologies. Design or implement complex algorithms requiring adherence to strict timing, system resource, or interface constraints. Perform quality control on team products. Recommend and implement suggestions for improving documentation and software development process standards. Oversee one or more software development teams and ensure the work is completed in accordance with the constraints of the software development process being used on any particular project. Confer with system engineers and hardware engineers to derive software requirements and to obtain information on project limitations and capabilities, performance requirements, and interfaces. Coordinate software installation on a system and monitor performance to ensure operational specifications are met. Recommend new technologies and processes for complex software projects. Serve as the technical lead of multiple software development teams. Select the software development process in coordination with the customer and system engineering. Ensure quality control of all developed and modified software. Delegate programming and testing responsibilities to one or more teams and monitor their performance. Requirements Experience using the Linux CLI and Linux tools Experience developing Bash scripts to automate manual processes Recent software development experience using Python and Java Experience using Apache Airflow (DAG design, scheduling, operators, sensors) to orchestrate, schedule, and monitor complex workflows Experience with Distributed Big Data processing engines including Apache Spark Familiar with SQL technologies such as MySQL, MariaDB, and PostgreSQL for querying, joining, and aggregating large datasets Experience using Jupyter Notebook Experience with data wrangling and preprocessing using tools such as pandas, NumPy Experience working with structured, semi-structured, and unstructured data such as Parquet, JSON, CSV, XML Familiarity with data quality concepts, data validation, and anomaly detection Experience with Git Source Control System Desired Skills Familiar with HPC Job Scheduling tools including Slurm Experience using the Atlassian Tool Suite (JIRA, Confluence) Qualifications Master’s degree in computer science or related discipline from an accredited college or university, plus five (5) years of experience as a SWE, in programs and contracts of similar scope, type, and complexity. OR Bachelor’s degree in computer science or related discipline from an accredited college or university, plus seven (7) years of experience as a SWE, in programs and contracts of similar scope, type, and complexity OR Nine (9) years of experience as a SWE, in programs and contracts of similar scope, type, and complexity. Benefits CareFirst Insurance Medical Plans Three available medical plans–2 Platinum PPO Plans ($0 Deductible) and 1 HSA Plan ($1500 deductible) Halogen pays 100% Medical Premium Costs up to $25,000 CareFirst Wellness Rewards Program Dental Plan Halogen pays 50% of the plan costs Vision Plan Halogen pays 100% of the plan costs 401K Retirement Plan · 10% contribution paid by the company, automatically vested and paid in each pay period. Life Insurance, AD&D, Short and Long Term Disability Coverage Paid Time Off · 20 Paid Time Off Days · 6 Federal Holidays · 5 Floating Holidays Bonuses · Based on performance and billable hours Halogen Engineering Group does not discriminate in employment on the basis of race, color, religion, sex (including pregnancy and gender identity), national origin, political affiliation, sexual orientation, marital status, disability, genetic information, age, membership in an employee organization, retaliation, parental status, military service, or other non-merit factor.