Linux: system administration, LFCS preparation

The steps

1. Step 1
   Beginner

   Linux: inventory a server (services, ports, logs)

   The contractor who ran the ERP server left three weeks ago, without handing anything over. Nobody knows what runs on it, or what would come back on its own after a power cut. Management is not asking you to fix anything: they want a written inventory, so that the next incident is not handled blind. You start from a shell and nothing else. It is the situation almost every system administrator starts with, and the one no course prepares you for. The guide does not give you the answers: it gives you the questions and the tools, and you search the machine before opening the solution. You will learn to read the file tree to separate what a package installed from what a human put there by hand, to ask systemd which unit a process belongs to, and to put a name on every listening port. Along the way, you will come across what every inherited server hides: business data no standard backup would take, a program nobody watches, and a script that works by hand but would fail as a service. You fix nothing, you document. Your inventory is read on the machine at the end of the lab, and it counts for your ranking. A real server, with its services in production and its logs. The terminal opens in your browser: no SSH key, nothing to install.

   • No reviews yet
   • Beginner
   • 1 h 45 min
   • 500 credits
   • First lab free
2. Step 2
   Beginner

   Linux: users, groups and permissions

   Two operations staff and an accountant start on Monday. Today everyone logs in with the same admin account, whose password is written on a sticky note. The statutory auditor has set three conditions before signing off the year: named accounts, access to the accounting records limited to the people concerned, and traceability of administrative actions. You have the weekend. Managing rights is what gets handed to a junior administrator first, and what is most often done wrong: a chmod 777 that solves today's problem and opens this year's. You will start with the audit, and it holds a surprise: a former employee has kept full powers. Then you will translate the business need into groups, build a share where every new file stays editable by the whole team (group, setgid, umask), and grant one precise command through sudo instead of root power. The server is in production, and the order of your moves matters as much as the moves themselves: build, prove under each newcomer's identity, and only then close the old accesses. The guide makes you predict the result before every command, because the gap between what you expected and what you see is what makes Unix permissions click. At the end, the machine's state is checked and counts for your ranking. Sandbox ready, terminal in the browser, no key to manage.

   • No reviews yet
   • Beginner
   • 2 h 30 min
   • 750 credits
   • First lab free
3. Step 3
   General

   Linux: patch a vulnerability with apt and dnf (Debian, RHEL)

   A critical vulnerability has just been published in GnuTLS, the encryption library present across your whole estate, and it must be fixed within 24 hours. Except that the ERP vendor wrote to you this morning: their connector is only certified with the version of curl currently installed. Any curl upgrade stops invoicing. So you have to fix the vulnerability without letting curl move, on both halves of the estate, and be able to justify it to the auditor. A uniform estate only exists in training courses. Here you work with both families side by side, a server from the RHEL family and a Debian 12 container stuck at its 2023 version, and you learn to translate every operation from one to the other: dpkg and rpm to know, apt and dnf to change. First trap, the one that trips up most beginners: the package carries neither the command's name nor the same name from one distribution to the other. You will look at what an update would take with it before running it, fix only what is necessary and prove it by the installed version, not by the absence of errors. Then you will freeze curl on both sides, try to force the lock, and document the debt this freeze creates, with its lift date. That last point is the one you will be blamed for if it is missing. At the end, the machine and your sheet are checked, and the result counts for your ranking. Terminal in the browser, nothing to install.

   • No reviews yet
   • Intermediate
   • 2 h 50 min
   • 1,000 credits
   • First lab free
4. Step 4
   General

   Linux: full disk and growing an LVM volume

   3:12 in the morning. The on-call phone rings: not a single invoice is going out, the /srv/factures volume is at 100%. You must restore the service tonight, without rebooting the server, without making /etc/fstab invalid, and without deleting anything that has accounting value. In the morning, management will ask one question only: why was nobody warned before the outage? “Disk full” covers three different failures, and they are not treated the same way. The guide makes you diagnose before you touch: you predict what df and du will answer, and their disagreement leads you to a file deleted the day before but still held open by a service, which still takes up hundreds of megabytes. You give that space back without erasing a single file, then you grow the logical volume and its file system online, without unmounting and without interrupting invoicing. The lab then goes where tutorials stop. Deciding whether swap is a cure or a symptom by reading real memory usage, rather than following a ready-made rule. Then setting up what was really missing: a measurement of how full the disk is, scheduled by a systemd timer, and a CloudWatch alarm with a threshold that leaves time to act during the day. At the end, the machine's state and the alarm are checked, and the result counts for your ranking. A real server, in incident conditions.

   • No reviews yet
   • Intermediate
   • 3 h 15 min
   • 1,500 credits
5. Step 5
   General

   Linux networking: routing and the nftables firewall

   The company switched Internet providers two days ago. Since then, customers can no longer track their containers online, while at the office the portal displays perfectly. The provider answered your ticket within an hour: the problem is not on their side. Management gives you the morning to give customers their portal back, and to answer one question in writing: who was right? You start by narrowing down the outage instead of searching at random: the same test from the office and from the Internet rules out half the suspects. Then a packet capture settles the argument with the provider, with evidence. You then go down layer by layer: the address, the route, the neighbours, and a gateway that has not existed since the router was replaced. You learn to recognise, from the shape of the error alone, a service that is not listening from a firewall that filters, and you catch the firewall in the act in the kernel log. The lab goes where tutorials stop. Understanding why a rule that was wrong from day one worked for years. Opening exactly one port in nftables, without opening administration to the Internet or shutting out the office. Then making every fix permanent, and proving it without rebooting, before writing the incident report that management and the provider will read. The office, the provider's router and a customer on the Internet are simulated on the machine: you can break the route and the firewall without losing your terminal. At the end, the server's state and your report are checked, and the result counts for your ranking.

   • No reviews yet
   • Intermediate
   • 2 h 50 min
   • 1,500 credits
6. Step 6
   Advanced

   Linux: systemd services, timers and udev rules

   Customs write for the third time: not a single declaration has reached the single window in three days, and containers are stuck at the port. The same morning, the dock supervisor reports that the label printer has printed nothing since the dock workstation was plugged back in. In lab 1, you had put the collection script in the risks column: started at boot by cron, detached by nohup, its output thrown into /dev/null. The risk has materialised. You start with the investigation: the log that stops without a word, the UPS log revealing that only power cuts ever restarted the collection, then the script run by hand that dies before your eyes at the first refusal from the customs window. Before touching anything, you put the configuration under Git. You then hand the collection over to systemd: a unit that restarts it without wearing itself out, under its own account, with bounded memory and a journal where nothing gets lost any more. The nightly summary, which cron forgot on power-cut nights, becomes a timer that catches up on missed runs. That leaves the barcode scanner. The kernel numbers devices in the order they arrive, and the labelling software now listens to the keyboard. You read what the kernel and udev know about a device, meet the modules that drive it, and write the udev rule that names it by what it is, not by its place in the queue. To finish, a colleague "simplifies" the configuration on a Friday evening, and Git tells you in ten seconds what changed. The customs window and the dock hardware are simulated on the machine, and behave like the real ones. At the end, the server's state and your Git history are checked, and the result counts for your ranking.

   • No reviews yet
   • Intermediate
   • 3 h
   • 1,500 credits
7. Step 7
   Advanced

   Linux: centralise accounts with LDAP, SSSD and PAM

   The statutory auditor is back. In lab 2, you locked the account of ktraore, the dock agent who left months ago. The preliminary report fits in one sentence: ktraore is locked on srv-erp, but active on srv-quai, with administration rights, and the auditor logged in with a password found in the former contractor's notes. The group has had a central directory since last year, which nobody finished connecting. The follow-up visit is on Friday. You start by mapping the estate: the same employee carrying two different numbers from one server to the next, and a local account silently inheriting the directory's groups. You learn to read an LDAP directory, its branches, entries and filters, encrypting the connection and introducing yourself with an account that can only read. Then you disable the former employee in the directory, and discover that a central revocation is worth nothing as long as a local account carries the same name. The lab goes where tutorials stop. Connecting a server with SSSD, NSS and PAM, and understanding why SSSD refuses to start with a file readable by everyone. Migrating local accounts to the directory without any file changing hands, knowing why a chown run on the whole disk would steal their files from a container's accounts. Seeing that a badly chosen order reopens the door you thought was closed. Cutting the directory, seeing who can still get in thanks to the cache, and testing the emergency account before you need it. Finally, writing the departure procedure the auditor will read on Friday. The directory is a real OpenLDAP, and the dock server a container with its own SSSD: you work on a small estate, from a single terminal. At the end, real login attempts check your revocations, as the auditor will, and the result counts for your ranking.

   • No reviews yet
   • Advanced
   • 3 h 15 min
   • 1,500 credits
8. Step 8
   Advanced

   Linux: SELinux and rescuing a server that won't boot

   The security audit comes down to two findings. On srv-portail, the server that gives customers the container tracking portal, SELinux has protected nothing since the contractor set it to permissive mode, "because the portal would not start with it". And nobody has ever tested a reboot: since last night's power cut, srv-archives, which keeps the customs declarations, no longer boots. You first read the state of SELinux, the contexts of processes and files, and the audit log, where permissive mode noted every denial without enforcing any. You switch to enforcing to see the portal go down, then rule out the three false fixes that bring the portal back by opening far more than the need: switching SELinux off, the boolean audit2why suggests, and the module audit2allow writes. The real fix comes down to two labels: port 8090 and the portal's folder, with semanage and restorecon. SELinux goes to enforcing mode, now and at boot, with no loss of service. AppArmor, the module on Debian and Ubuntu, is placed alongside. Then the second server: in emergency mode, root locked, no way in. You read its console, find the failing mount, stop the server, check its disk, mount it from the outside as from a rescue instance, fix the typo in its fstab and test it with a test mount. Finally the proof: you find out why the server, although "enabled", did not restart on its own, and you really reboot the machine. A written report closes the audit. The last lab of the path: it covers the last two chapters of LFS207, Linux security modules and system rescue.

   • No reviews yet
   • Advanced
   • 3 h
   • 1,500 credits

The Linux Foundation, LFCS and the associated badge are trademarks of The Linux Foundation. Linux is a registered trademark of Linus Torvalds. MonPremierLab is not affiliated with the Linux Foundation.