Senior Linux Infrastructure Engineer, tastytrade

• 6+ years in a Linux systems, infrastructure, or SRE role
• Performance analysis. You can go from a vague complaint to a named subsystem using the standard tooling — perf, strace, ss, iostat, bpftrace, or equivalents — and explain what the numbers mean
• Troubleshooting methodology. A disciplined, hypothesis-driven approach that works on a system you have never seen before. We care more about how you narrow the problem than about which commands you happen to know
• Systems fundamentals. Processes and signals, systemd, cgroups and namespaces, filesystems, and what actually happens when a host exhausts memory or file descriptors
• Declarative configuration management at production scale — Salt, Ansible, Chef, or Puppet. You have authored and maintained the code, not only run it
• Containerization and orchestration. Production experience with Kubernetes or Nomad, including the operational realities: scheduling, resource pressure, rollouts, and debugging a workload that will not start
• Scripting and automation. Strong Bash and working Python. You write code that other engineers can maintain
• Foundational networking. Comfortable across layers — VLANs and routing, DNS, DHCP, TCP behavior, and TLS — and able to determine whether a problem is the application, the host, or the network
• Proxy and load-balancing experience with Nginx or HAProxy
• Virtualization experience with at least one of VMware, Xen, or KVM
• Git and peer review as a daily habit, including the discipline to keep changes reviewable
• Excellent written communication. Runbooks, design proposals, and incident write-ups are a core part of this job, not an afterthought
• The ability to get productive quickly in areas where you do not yet have depth
• If you’re excited about this role, but your experience doesn’t align perfectly, we encourage you to apply anyway

Desirable

• We are looking for a senior Linux engineer who is equal parts operator and builder. You will own the systems layer of our production environment — from the metal to the configuration management code. This is a hands-on role: you will jump into incidents, tackle challenging engineering problems, design systems that scale, and evaluate new software and architectures
• This is not a role where the abstractions hide the machine. You are expected to be curious about what is underneath and to be the person other engineers come to when a system is behaving in a way nobody can explain
• Own Linux performance. Diagnose and tune systems under real production load: CPU scheduling and NUMA placement, memory and page cache behavior, disk and filesystem I/O, and network stack tuning
• Lead troubleshooting on hard problems. Work incidents methodically — form a hypothesis, find the cheapest test that falsifies it, and narrow the search rather than changing five things at once. Write postmortems, organize follow-up tasks, and future-proof the environment so the same issue does not recur
• Write and maintain configuration management code. Build infrastructure declaratively with Salt, Ansible, Chef, or Puppet, treating that code with the same standards as application code: reviewed, tested, and version-controlled
• Run containerized workloads. Build, deploy, and operate services on Kubernetes or Nomad, including scheduling behavior, resource limits, health checking, and the failure modes that only appear under contention
• Automate in Bash and Python. Replace manual runbooks with tooling and streamline repeatable work
• Operate core network services. Troubleshoot TCP/IP with confidence and bring solid networking fundamentals. Manage DNS and DHCP as production services — zone management, resolver behavior, TTL strategy, scopes, reservations, and relay configuration
• Operate the traffic and data tier. Configure and troubleshoot Nginx and HAProxy (routing, TLS termination, health checks, connection handling) and support Redis and RabbitMQ in production
• Manage virtualization. Provision and maintain guests across VMware, Xen, or KVM, including capacity planning, host maintenance, and live migration
• Handle secrets properly. Use Vault for secret storage, dynamic credentials, policy, and rotation — and help move the organization off whatever it was doing before
• Own observability. Maintain log aggregation on the Elastic Stack and alerting through Nagios, CheckMK, or Icinga. Tune alerts toward signal; an alert nobody can act on is a bug
• Work through change management and code review. Everything moves through Git and pull requests. You will review other people's changes as seriously as you expect yours to be reviewed