Stable log references.
inodes, hard links, and dead symlinks.

logadmin1 needs a reference to the current audit log that survives rotation, and a way to catch the broken symlinks that went unnoticed for two days. This is inodes, hard links versus soft links, and find.

The ticket: logadmin1 has an audit log whose name keeps changing because of rotation. They want a stable reference to the current log that does not break when rotation renames the file, and the backup service needs the same thing. On top of that, last week someone deleted a file and the symlink pointing at it broke silently for two days. They want to know how to catch that.

The real problem: A filename is not a file. Once that idea lands, hard links and soft links stop being confusing and the whole ticket falls out of it.

What we are doing: Learning inodes, hard links, and symbolic links on real files, then building a proactive check for dead symlinks.

The ticket

Confirm the users the ticket references exist before you touch anything.

id logadmin1   # UID 8001
id svc_auto    # the backup proxy account, UID 6503

Concept review. What a filename really is.

Read the manual pages first. These four are the whole toolkit for this ticket.

man ln
man find
man stat
man readlink

An inode is the file. It holds the permissions, the owner, and the pointers to the data blocks. A filename is just a label that points at an inode. That single fact explains both link types.

Hard link: another name for the same inode. Same inode number, same data. It survives the original name being deleted or renamed, because it was never pointing at the name, only at the inode. Hard links cannot cross filesystems and cannot point at a directory.

Soft link, or symlink: a tiny file that stores a path string. It has its own inode. If the path it stores is moved or deleted, the symlink is left pointing at nothing. Symlinks can cross filesystems freely and can point at directories.

The hard link. A reference that survives rotation.

Set the scene, then link it. ls -li prints the inode number first so you can prove the two names are the same file.

sudo mkdir -p /srv/logs/audit
sudo bash -c 'echo "audit event 1" > /srv/logs/audit/audit.log.20260503'
sudo chown -R logadmin1:logadmins /srv/logs/audit
ls -li /srv/logs/audit/

# create a hard link to give logadmin1 a stable name
sudo ln /srv/logs/audit/audit.log.20260503 /srv/logs/audit/audit.current
ls -li /srv/logs/audit/    # same inode number for both

Now simulate rotation by renaming the original, then read through the stable name.

sudo mv /srv/logs/audit/audit.log.20260503 /srv/logs/audit/audit.log.20260503.gz
cat /srv/logs/audit/audit.current    # still works
ls -li /srv/logs/audit/

The hard link followed the inode, not the name, and the inode did not move. That is the stable reference logadmin1 asked for.

The symlink. A reference that can rot.

A symlink is the right tool when you need to cross a filesystem or point at a directory. It is also brittle, and this is exactly the failure that bit them.

sudo ln -s /srv/logs/audit/audit.current /srv/logs/audit/backup.target
readlink /srv/logs/audit/backup.target    # shows the stored path
cat /srv/logs/audit/backup.target         # works, for now

# delete the target and watch the link go dead
sudo rm /srv/logs/audit/audit.current
ls -l /srv/logs/audit/backup.target       # shown in red, dangling
cat /srv/logs/audit/backup.target         # No such file or directory

The link still exists. It just points at nothing. Nobody notices until they try to read it, which is how it went unseen for two days.

Detecting broken symlinks

find has a filter built for this. -xtype l matches symlinks whose target does not resolve. Run it on a schedule and dead links surface the same day they break.

# every broken symlink under /srv
sudo find /srv -xtype l 2>/dev/null

# turn it into a report for logadmin1
sudo mkdir -p /srv/audit_reports
sudo find /srv -xtype l 2>/dev/null > /srv/audit_reports/broken_symlinks.txt
cat /srv/audit_reports/broken_symlinks.txt

cp versus ln, made concrete

Worth seeing once so it never trips you up. cp makes a new inode, an independent copy. ln makes another name for the same inode.

echo "original content" | sudo tee /srv/logs/audit/source.log
sudo cp /srv/logs/audit/source.log /srv/logs/audit/copy.log      # new inode
sudo ln /srv/logs/audit/source.log /srv/logs/audit/hardlink.log  # same inode
sudo ls -li /srv/logs/audit/source.log /srv/logs/audit/copy.log /srv/logs/audit/hardlink.log

# edit the source and see who changes
sudo bash -c 'echo "new line" >> /srv/logs/audit/source.log'
sudo cat /srv/logs/audit/copy.log       # unchanged, separate file
sudo cat /srv/logs/audit/hardlink.log   # has the new line, same file

Exam questions

Write the command before you run it.

Q1. Create a hard link for /etc/hostname at /srv/logs/audit/hostname_hard and prove they share an inode.

Q2. Create a symbolic link for /var/log at /srv/logs/audit/varlog_link and confirm its target with readlink.

Q3. Find all broken symlinks under /srv and save the list to /srv/audit_reports/broken_links.txt.

Answers.

# A1
sudo ln /etc/hostname /srv/logs/audit/hostname_hard
ls -li /etc/hostname /srv/logs/audit/hostname_hard    # inode numbers match
# A2
sudo ln -s /var/log /srv/logs/audit/varlog_link
readlink /srv/logs/audit/varlog_link
# A3
sudo find /srv -xtype l 2>/dev/null > /srv/audit_reports/broken_links.txt

Final checklist: confirm everything works

If every check passes, the ticket is done.

# 1. hard link shares the inode of its target
ls -li /srv/logs/audit/audit.current

# 2. broken symlink report exists
cat /srv/audit_reports/broken_symlinks.txt

# 3. readlink shows a symlink's target
readlink /srv/logs/audit/backup.target

# 4. eyeball: all three exam commands written from scratch
# 5. tracker entry checked off

Reply to logadmin1: use a hard link for the stable reference, it tracks the inode so it survives rotation. Use a symlink only when you need to cross a filesystem or point at a directory, and run find /srv -xtype l weekly so a dead link never goes unseen for two days again.