Deploy a Secure Web Server with Persistent Storage

Traditional disk partitions are fixed at creation. Size /var/www/html at 2 GB today and need 20 GB next quarter, and a raw partition forces a backup-repartition-restore cycle. LVM (Logical Volume Manager) inserts a layer of indirection between the physical disk and the filesystem that makes storage elastic, which is exactly why the exam leans on it.

The stack has three tiers. A physical volume (PV) is a disk or partition handed to LVM with pvcreate. One or more PVs are pooled into a volume group (VG) with vgcreate, think of the VG as a single allocatable capacity pool. From that pool you carve logical volumes (LVs) with lvcreate; an LV behaves like a partition but can be grown, and in some cases shrunk or moved, while mounted. The kernel exposes each LV through device-mapper at /dev/mapper/vg_web-lv_webdata, with a convenience symlink at /dev/vg_web/lv_webdata. Capacity is tracked in fixed-size physical extents (4 MiB by default), which is why volumes always round to extent boundaries.

XFS is the RHEL 9 default filesystem, a high-performance journaling filesystem built for large volumes and heavy parallel I/O. One exam-critical quirk: XFS can be grown online with xfs_growfs but can never be shrunk. If a task ever says “reduce the filesystem,” that is ext4 territory, not XFS.

Persistence lives in /etc/fstab, which systemd reads at every boot to assemble the mount tree. The golden rule is to mount by UUID, never by device path: kernel names like /dev/sdb are assigned in device-discovery order and can shift when disks are added or reordered, whereas a filesystem UUID is written into the superblock and is stable for the life of the filesystem. The six fstab fields are device, mount point, type, options, dump, and pass. defaults 0 0 means standard options, no dump-based backup, and no boot-time fsck, XFS performs its consistency checks online, so the pass field is 0.

The cost of an fstab mistake is severe and is a classic way to fail the exam: a single bad entry can drop the system into emergency mode on the next boot. That is why you always run mount -a immediately after editing, it mounts everything declared in fstab and surfaces syntax or UUID errors now, while they are trivial to fix, rather than at the reboot when they are not.

Linux identity is numeric. Every user has a UID and every group a GID; the names you see are a convenience layer resolved through /etc/passwd and /etc/group. By convention, UIDs and GIDs below 1000 are reserved for the system and services, while regular human accounts start at 1000. Choosing 6000/6001 here deliberately places this service identity in the “service” range, well clear of real users.

A user has exactly one primary group (the GID recorded in /etc/passwd, applied to files they create) and any number of supplementary groups. useradd -g webteam sets the primary group explicitly; omit it and useradd helpfully creates a brand-new private group named after the user and ignores your intended GID scheme. The login shell matters too: /sbin/nologin gives the account no interactive shell, so it can own files and run services but cannot be logged into, the correct posture for a service account that should never have a human at a prompt.

Standard permissions are the familiar three triads, owner, group, other, each carrying read (4), write (2), and execute (1). On top of those sit three special bits, expressed as a leading fourth octal digit: setuid (4), setgid (2), and the sticky bit (1). The task uses setgid on a directory, which is the conceptual heart of this exercise. A setgid directory forces every file and subdirectory created inside it to inherit the directory’s group rather than the creating user’s primary group.

That is why 2775 is precise, not arbitrary. The leading 2 is setgid, guaranteeing new content stays owned by webteam no matter who writes it. 7 for owner and 7 for group give the team full read/write/execute. The final 5, read and execute for others, is what lets the nginx worker process, which runs as an unprivileged user outside webteam, traverse the directory and read the files it serves. Drop that to 2770 and nginx returns 403s. Set ownership with chown before chmod, because the setgid bit binds to whatever group owns the directory at the moment it is set.

This task chains three subsystems: package management, service management, and the firewall. Packages come from dnf, RHEL’s dependency-resolving front end to the RPM database; dnf install pulls nginx and everything it needs from the configured repositories. Service lifecycle is systemd’s job: systemctl start runs a unit now, systemctl enable wires it into the boot sequence, and —now does both in one command. On the exam, “enabled and started” almost always means enable —now, forgetting enable is how a service that worked in your session vanishes after the reboot check.

One detail the steps below quietly depend on: nginx’s default document root on RHEL is /usr/share/nginx/html, not /var/www/html, the latter is Apache’s historical convention. For curl localhost to return your page, nginx must actually be pointed at /var/www/html: set root /var/www/html; in the server block of /etc/nginx/nginx.conf (then nginx -t && systemctl restart nginx). Skip that and nginx cheerfully serves its default welcome page, and the SELinux context you set in Task 4 will look like the culprit when it is innocent. This is one of the most common “it should work but doesn’t” traps in the whole project.

firewalld is RHEL’s default firewall manager, and it is zone-based: every interface belongs to a zone (usually public), and each zone has a rule set. Rules exist in two layers, a runtime configuration that is live but volatile, and a permanent configuration written to disk. —permanent writes the rule but does not apply it live; —reload re-reads the permanent config into runtime. The reliable pattern is therefore —permanent then —reload; using neither means the rule evaporates at reboot, the single most common firewall mistake on the exam. The named services http and https are simply predefined bundles mapping to TCP 80 and 443.

SELinux is mandatory access control layered on top of ordinary Unix permissions. Even when the file mode says “allowed,” SELinux can still deny the access if the policy disagrees, which is precisely why a perfectly readable file can still produce a 403 from nginx. The mechanism is type enforcement: every process runs in a domain and every file carries a type, and the policy defines which domains may touch which types.

A context has four colon-separated fields, SELinux user, role, type, and level, but for file labeling the type is the field that decides everything. Web content must carry httpd_sys_content_t, because the web server domain (httpd_t, which nginx shares on RHEL) is permitted to read exactly that type. The default policy already labels /usr/share/nginx/html and /var/www correctly; the moment you serve content from a freshly mounted LVM volume, that new filesystem has no such labeling and you must establish it yourself.

There are two ways to set a context, and choosing correctly is the exam point. chcon changes the label on disk immediately but does not record it in policy, so the next filesystem relabel, or a stray restorecon, silently reverts it. semanage fcontext -a writes a rule into the local policy describing what the label should be, making the change durable; restorecon then reads that policy and applies the correct label to files on disk. The persistent, correct workflow is always semanage fcontext followed by restorecon.

The regular-expression suffix (/.*)? is essential and frequently forgotten. It means “this directory and everything beneath it.” Without it the rule labels only the directory node and leaves the files inside unlabeled, which produces the maddening symptom of a correctly contexted directory that still cannot be served. Finally, verify with getenforce: if SELinux is Permissive, denials are logged but not enforced, so a working curl proves nothing, the misconfiguration is waiting to bite the instant the system returns to Enforcing.

Hardening SSH is where a careless command turns into a locked-out host, so the order of operations is the lesson. Three directives in /etc/ssh/sshd_config do the work. PermitRootLogin no blocks direct root authentication, forcing administrators to log in as a normal user and escalate, which preserves an audit trail and removes the single most attacked username on the internet. PasswordAuthentication no disables password logins entirely, defeating brute-force and credential-stuffing attacks by requiring cryptographic keys. AllowGroups webteam is an allow-list: only members of webteam may authenticate at all, regardless of other settings.

The danger is that AllowGroups takes effect the instant sshd restarts. If your own account is not already in webteam, the restart severs every future SSH session, including yours. That is why the very first step adds your user to the group, and why disabling password auth without a working key pair in place is the express route to a host you can only reach from the physical console. Key-based auth relies on a private key on your client and its matching public key in ~/.ssh/authorized_keys on the server; the file must be mode 600 (and ~/.ssh mode 700), because sshd refuses overly permissive key files as a security measure.

The professional safeguard is sshd -t, which parses the configuration and validates syntax without touching the running daemon. Run it before every restart: a typo caught by sshd -t costs you ten seconds, whereas the same typo discovered after a restart can leave sshd refusing to start and you staring at a console login at the worst possible time.

This task is a compact lesson in defensive shell scripting. The shebang #!/bin/bash on line one tells the kernel which interpreter to use; without it the script may run under whatever shell happens to invoke it, with subtly different behavior. Command substitution, $(date +%Y%m%d), captures the output of a command into a variable, here producing a dated filename like web_20260420.tar.gz so each run is uniquely named rather than overwriting the last.

The script’s correctness hinges on checking the exit status. Every command returns a status code: 0 for success, non-zero for failure. Wrapping tar in if tar …; then branches on that status, so the script reports “Backup created” only when the archive truly succeeded and exits 1 on failure, a non-zero exit is what lets schedulers and monitoring detect that the job broke. tar -czf means create, gzip-compress, and write to the named file; the 2>/dev/null discards tar’s harmless “removing leading /” notice on stderr.

A subtle but important detail is the quoted heredoc, <<‘EOF’. Quoting the delimiter tells bash to write the body verbatim, so $DATE and $(date …) land in the file unexpanded and are evaluated each time the script runs. An unquoted <<EOF would expand those variables once, at file-creation time, freezing the date forever, a classic, hard-to-spot bug. The closing rule of the task is operational, not syntactic: always run a new script by hand before scheduling it, because cron and systemd run it silently and a broken backup discovered months later is no backup at all.

Scheduling the same job two ways is deliberate: the exam wants proof you can drive both the classic and the modern scheduler. cron is the traditional time-based job runner. A crontab line has five time fields, minute, hour, day-of-month, month, day-of-week, followed by the command. 0 2 * * * reads as “minute 0 of hour 2, every day, every month, every weekday,” i.e. 2:00 AM nightly. The most common error is field-order confusion; 2 * * * * would instead run at two minutes past every hour.

The systemd approach splits the work across two units. A service unit defines what to run; here Type=oneshot is correct because the script runs to completion and exits, as opposed to Type=simple for long-lived daemons. A timer unit defines when, and is bound to the service it triggers. This timer uses monotonic triggers: OnBootSec=10min fires ten minutes after boot, and OnUnitActiveSec=6h fires every six hours after the previous activation. (The alternative, OnCalendar=, expresses wall-clock schedules like cron.)

Two operational rules trip people up. First, after writing or editing any unit file you must run systemctl daemon-reload so systemd re-reads its configuration, skip it and systemd behaves as though the unit does not exist. Second, you enable and start the timer, not the service: enabling the service would run the backup once at enable time and never again, while enabling the timer is what installs the recurring schedule. Confirm the result with systemctl list-timers, which shows the next scheduled firing.

Accurate time is infrastructure, not housekeeping. Kerberos authentication rejects requests whose clocks skew more than a few minutes, TLS certificate validation depends on the current time, and correlating logs across machines is impossible if their clocks disagree. RHEL 9 keeps time with chrony, whose daemon chronyd disciplines the system clock against network time sources defined in /etc/chrony.conf.

That file distinguishes a pool directive, a DNS name that resolves to many rotating servers, from a server directive naming one specific source. The task replaces the defaults with a single explicit server, so the existing pool (and any stray server) lines must be commented out first; leave them in and chrony averages your intended source against the ones it is still polling, muddying the result. The iburst option tells chrony to send a rapid initial burst of packets on startup so it can lock on in seconds rather than minutes.

Verification is its own skill. chronyc sources lists each source with a status glyph in the first column: ^* marks the currently selected, synchronized source; ^+ an acceptable alternate; and ^? a source not yet reachable. Seeing ^? right after a restart is normal, wait and re-check. Separately, timedatectl shows the timezone and whether NTP synchronization is active; the timezone itself is a property of the system set with timedatectl set-timezone, independent of the clock-sync source.

Podman is RHEL’s container engine, a drop-in alternative to Docker that runs without a central daemon and supports unprivileged “rootless” containers. Images follow the OCI standard; registry.access.redhat.com/ubi9/ubi-minimal is Red Hat’s Universal Base Image, a small, freely redistributable RHEL-derived image that needs no subscription to pull. podman run -d starts a container detached, and sleep infinity is the idiomatic way to keep an otherwise-idle container alive so it stays available for inspection.

The volume flag -v /var/log/nginx:/logs:Z bind-mounts a host directory into the container, and the trailing :Z is the SELinux-critical part. Containers run confined under the container_t domain, which cannot read arbitrary host files. :Z tells Podman to relabel the host directory with a private container context so the container may access it; a lowercase :z applies a shared label instead. Omit the suffix entirely and SELinux denies the mount’s contents, producing the illusion of a broken volume.

Making the container start at boot means handing its lifecycle to systemd. The walkthrough uses podman generate systemd —new, which emits a unit file describing the container; the —new flag is essential because it makes the unit recreate the container from its definition rather than referencing a one-off container ID that disappears if the container is ever removed. Note for currency: podman generate systemd is deprecated on RHEL 9 in favor of Quadlet, where you describe the container in a .container file under /etc/containers/systemd/ and systemd generates the unit at daemon-reload. The generate-systemd method still works and is what many current exam objectives reference, but expect Quadlet to be the forward path.

The systemd journal (systemd-journald) is the central, structured log store for the whole system. By default on many installs it is volatile: logs live in /run/log/journal, which is a tmpfs in RAM, so every reboot wipes the history, fine for a throwaway box, useless for diagnosing why a server crashed last night. Switching to persistent storage keeps logs across reboots in /var/log/journal.

The configuration has a deceptively simple trap. Setting Storage=persistent in /etc/systemd/journald.conf is necessary but not sufficient: journald only writes persistently if the directory /var/log/journal actually exists. Create the directory, restart journald, and persistence engages; set the option but skip the directory and the journal silently keeps writing to RAM. SystemMaxUse=300M caps the on-disk journal so logs cannot fill the partition, an important safety valve on production systems.

The boot target is a separate concept that rides along in this task. systemd targets are named groupings of units that represent a system state; multi-user.target is a full multi-user system without a graphical desktop, while graphical.target adds the display manager. The default is a symlink, /etc/systemd/system/default.target, and you set it with systemctl set-default, which is persistent. Do not confuse it with systemctl isolate, which switches the current session’s target immediately but reverts at the next boot; using isolate when the task wants a persistent change is a frequent mistake.

autofs mounts filesystems on demand and unmounts them after a period of inactivity, rather than holding them mounted continuously the way an /etc/fstab entry would. For network shares this is a meaningful advantage: the mount is attempted only when something actually accesses the path, so a temporarily unreachable NFS server cannot hang the boot process or wedge a static mount, and idle shares release automatically.

Configuration is split across two kinds of file. The master map, on RHEL 9, a drop-in under /etc/auto.master.d/, associates a base mount point with a map file: /mnt /etc/auto.content says “keys defined in /etc/auto.content are mounted under /mnt.” The referenced map file then defines the actual mounts. The entry content -rw,sync fileserver…:/exports/content is an indirect map: the key content becomes the subdirectory /mnt/content, created and mounted only when accessed. (A direct map, by contrast, uses an absolute path as the key and a /- placeholder in the master map.)

The behavior that confuses newcomers is that /mnt/content does not visibly exist until something touches it, listing /mnt alone will not show it and will not trigger the mount. You must access the full path, e.g. ls /mnt/content, to fire the automounter. Note that in this lab the NFS server is fictional, so the autofs configuration can be entirely correct while the mount itself stays empty; on the exam you are graded on the configuration, not on a live remote export.

The final task encodes the single most important habit the whole project is teaching: nothing counts until it survives a reboot. Every preceding task had a persistence dimension, —permanent on firewalld, enable on every service and timer, set-default for the boot target, UUID entries in fstab, semanage for SELinux, a Quadlet or generated unit for the container. A configuration that works only in your live session is a magic trick; the reboot is what separates a trick from a real, durable system.

Reboot deliberately, then walk the verification checklist line by line. Each command confirms one task’s persistence: the volume re-mounted, the service active, the page served, the context intact, the timer armed, the clock synced. If any line returns the wrong result, the fault is almost always a missing persistence step, an un-enabled unit, a runtime-only firewall rule, an isolate where set-default belonged. The troubleshooting table below maps the common post-reboot symptoms to their usual causes so you can repair rather than guess.