hp-chell/NEON-CUSTOMIZATIONS.md

# Other neon customizations on hp-chell

Reconstructed on 2026-09-27 from `~matt/.bash_history` (551 lines,
roughly early March 2026 onward), `/root/.bash_history`, and the live
system. Topics with their own docs:
[`AUDIO.md`](AUDIO.md), [`ZRAM.md`](ZRAM.md),
[`CHROMEOS-FLEX.md`](CHROMEOS-FLEX.md), and the kernel 7.x black-screen
fix in `CLAUDE.md`.

## Keyboard: keyd remapping for the Chromebook layout

The Chromebook keyboard has no F-keys, Delete, Home/End, PgUp/PgDn or
Caps Lock; there's a "Search" key where Caps Lock would be. **keyd**
remaps it at the evdev level, so it works everywhere (console, X,
Wayland, the login screen).

- Built from source, not from apt (apt has no keyd for noble):
  `~/github/keyd` (<https://github.com/rvaiya/keyd>, `7c0aecb`,
  **v2.6.0**), `make && sudo make install`, which puts the binary at
  `/usr/local/bin/keyd`. Then `sudo systemctl enable --now keyd`. Neither
  apt nor unattended upgrades will ever update it; rebuild from the
  checkout to upgrade.
- Config: `/etc/keyd/default.conf`. Apply edits with `sudo keyd reload`;
  find key names and device IDs with `sudo keyd monitor`. Summary:
  - `[ids] 0001:0001`: **only the internal keyboard** (`AT Translated Set
    2 keyboard`); external USB keyboards are left alone (use `*` to
    include them).
  - `leftmeta = overload(meta, leftmeta)`: Search acts as Meta when held
    and taps as Meta (opens the KDE launcher).
  - **Search + top row gives F1 to F10** (back, forward, refresh, zoom,
    scale, brightness down/up, mute, volume down/up). **Search + `-`/`=`
    gives F11/F12.**
  - The same top-row to F-key mapping with Alt, Ctrl or Shift held. The
    `A-`/`C-`/`S-` prefixes re-apply the modifier, which keyd otherwise
    strips, so e.g. Alt+F4 works as Alt+Search+Zoom.
  - Search + Backspace = Delete; Search + arrows = Home/End/PgUp/PgDn.
  - Alt + Backspace = Delete; **Alt + Search = Caps Lock**.

## USB-C DisplayPort output: load the cros_ec Type-C drivers

`/etc/modules-load.d/cros-video.conf` (2026-03-12):

```
cros_ec_typec
typec_displayport
```

These don't autoload on chell, and without them the USB-C ports don't
negotiate DisplayPort alt mode, so there's no video out over USB-C.
Both are loaded as of 2026-09-27.

## Kernel: HWE-edge meta package

`linux-generic-hwe-24.04-edge` is installed (alongside the plain
`linux-generic-hwe-24.04`), and it's why the machine is on 7.x kernels
(`7.0.0-34` now), not noble's regular HWE line. Also installed
deliberately as a known-good fallback: `linux-image-generic-6.17` /
`linux-headers-generic-6.17` (`6.17.0-42`), which is the fallback kernel
named in the black-screen notes in `CLAUDE.md`. Old 7.x kernels
(`7.0.0-30`, `7.0.0-14`) are still installed as well.

`/etc/default/grub` (edited by hand several times; both `matt` and root
history show `nano /etc/default/grub` + `update-grub`): `GRUB_DEFAULT=0`,
`GRUB_TIMEOUT=5`, `GRUB_TIMEOUT_STYLE=menu`,
`GRUB_CMDLINE_LINUX_DEFAULT='splash resume=UUID=3af45daa-b8c2-4053-9574-a958e0c3ea68
resume_offset=17674240'` (the `resume` params were added 2026-09-28 for
hibernation, see below; see also `setup-i915-kernel7-fix.sh`).

## Storage / wear on the microSD root

neon's root is a microSD card in the built-in slot (`sda`; the slot's
reader is USB-attached internally), and microSD handles lots of small
writes poorly, so a few changes cut down writes and space use:
- **`/tmp` is tmpfs** (in `/etc/fstab`:
  `tmpfs /tmp tmpfs defaults,noatime,mode=1777 0 0`). Not in the shell
  history, so it was probably added in the same `nano /etc/fstab` edit
  that disabled the swapfile.
- The swapfile was replaced by zram (see `ZRAM.md`).
- **`localepurge`** is installed and configured (via `dpkg-reconfigure
  localepurge`) to keep only `en`, `en_CA.UTF-8`, `en_US.UTF-8` and
  `fr_CA.UTF-8`, and delete other locales' files on every package
  install (`USE_DPKG`, `MANDELETE`). If another language's translations
  or man pages are ever needed, add them with
  `sudo dpkg-reconfigure localepurge`.
- **Root mounted with `commit=30`** (2026-09-27): the root line in
  `/etc/fstab` is `defaults,commit=30`; the pre-change copy is
  `/etc/fstab.bak-2026-09-27`. ext4 commits its journal every 30s
  instead of every 5s, so writes are batched into fewer, larger bursts.
  Trade-off: a crash or forced power-off can lose up to ~30s of recent
  changes that weren't fsync'd (apt/dpkg, editors and databases fsync, so
  they're unaffected). 30 rather than 60 because this machine has needed
  forced power-offs. `relatime` is kept deliberately (not `noatime`).
- **journald capped at 100MB** (2026-09-27):
  `/etc/systemd/journald.conf.d/50-max-size.conf` sets
  `SystemMaxUse=100M` (it was 455MB).
- **`smartmontools` disabled** (2026-09-27, `systemctl disable --now`):
  it failed on every boot because neither the microSD card nor the eMMC
  supports SMART.
- **Marked non-rotational** (2026-09-27): the Genesys reader reported
  `rotational=1` (a spinning disk). `/etc/udev/rules.d/61-microsd-nonrotational.rules`
  sets `queue/rotational=0` for `ID_VENDOR_ID=05e3`, `ID_MODEL_ID=0747`,
  so `lsblk -d -o NAME,ROTA` shows `0`.
- The card doesn't support discard/TRIM (`lsblk -D` shows 0), so
  `fstrim.timer` does nothing for it. That's harmless and was left alone.

## Suspend: stay on `s2idle`; do NOT use S3 (`deep`)

Tested 2026-09-27. The firmware offers both `s2idle` (default) and `deep`
(ACPI S3) in `/sys/power/mem_sleep`.
- An RTC-woken 30s S3 test (`rtcwake -m mem -s 30` with `mem_sleep=deep`)
  entered and left S3 cleanly, and the display came back. The i915
  bug didn't show.
- **A real lid-close S3 suspend broke the root filesystem**: on resume
  the built-in microSD reader (USB, Genesys `05e3:0747`) didn't come
  back in time, the kernel logged disk I/O errors, and ext4 remounted `/`
  read-only (`errors=remount-ro`). Nothing was logged because root was
  read-only (the previous boot's journal just stops). After a forced
  reboot, the initramfs fsck replayed the journal and reported the
  filesystem clean; no lasting damage was found.
- In S3 the USB controller loses power, so the reader has to be
  re-attached. USB "persist" (`power/persist=1`, the kernel feature
  meant for root on USB storage) was already on and didn't save it.
  There's no dependable fix, so **never set `mem_sleep_default=deep`**
  while neon's root is on the microSD slot.
- `s2idle` keeps USB powered and has worked for months. **It does
  reach S0ix**: in a 44s lid-close test (2026-09-28),
  `/sys/kernel/debug/pmc_core/slp_s0_residency_usec` (needs root)
  grew by ~42.2s, and `package_cstate_show` showed Package C10 ~42.0s.
  So ~96% of the suspend was in the deepest idle state, and drain should
  be close to S3's. Root stayed `rw` with no I/O errors.
- Use the `pmc_core` counter above, not
  `/sys/devices/system/cpu/cpuidle/low_power_idle_system_residency_us`.
  That one comes from ACPI LPIT, which isn't wired up properly under
  coreboot here: it read 270µs for the same suspend.

## Suspend-then-hibernate (set up 2026-09-28)

A lid close suspends (`s2idle`, instant). If the machine is still asleep
after **3 hours**, systemd wakes it (RTC), writes a hibernation image to
disk and powers off. This protects against a dead battery without
writing a multi-GB image on every lid close, which is why this was
chosen over hybrid sleep. It doesn't need S3.

Pieces:
- **16GB `/swapfile`**, used for the hibernation image (sized to RAM for
  the worst case; a typical image is ~4GB).
  fstab `/swapfile none swap sw,pri=10 0 0`: priority 10 < zram's 100,
  so normal swapping still goes to zram. Backup of the old fstab:
  `/etc/fstab.bak-2026-09-28`.
- **Resume location**: `resume=UUID=3af45daa-b8c2-4053-9574-a958e0c3ea68
  resume_offset=17674240` in `GRUB_CMDLINE_LINUX_DEFAULT` (backup
  `/etc/default/grub.bak-2026-09-28`) and in
  `/etc/initramfs-tools/conf.d/resume`, then `update-initramfs -u -k all`
  + `update-grub`. **If `/swapfile` is ever recreated, its offset
  changes.** Get the new one with
  `sudo filefrag -v /swapfile | awk '$1=="0:"{print $4}'` (strip the
  trailing `..`), then update both places and rerun both commands.
- **`/etc/systemd/sleep.conf.d/50-suspend-then-hibernate.conf`**:
  `HibernateDelaySec=3h`.
- **`/etc/polkit-1/rules.d/10-enable-hibernate.rules`**: Ubuntu's
  `/usr/share/polkit-1/rules.d/com.ubuntu.desktop.rules` returns `NO` for
  all `login1`/`upower` hibernate actions, so logind reported
  `CanHibernate`/`CanSuspendThenHibernate` = `no` even though its sleep
  checks passed (seen with
  `systemctl service-log-level systemd-logind debug`). This rule sorts
  earlier and returns `YES` for local, active users.
  Ubuntu actually has *two* blockers: that JS rule and
  `/var/lib/polkit-1/localauthority/10-vendor.d/com.ubuntu.desktop.pkla`
  ("Disable hibernate by default in logind/upower"), which is still read
  because `polkitd-pkla` is installed. The `10-` rule overrides both:
  `busctl call org.freedesktop.login1 /org/freedesktop/login1
  org.freedesktop.login1.Manager CanSuspendThenHibernate` returns `yes`
  as `matt`.
- **Plasma**: PowerDevil owns the lid, so each Power Management profile's
  sleep mode is set to "Standby, then hibernate". That's `SleepMode=3`
  under `[AC|Battery|LowBattery][SuspendAndShutdown]` in
  `~/.config/powerdevilrc` (set 2026-09-28 for all three profiles).
  **Gotcha:** PowerDevil caches hibernate support when it starts, so
  after the polkit fix the option didn't appear in System Settings until
  `systemctl --user restart plasma-powerdevil.service` and System
  Settings was reopened. Check PowerDevil's view with
  `qdbus6 org.freedesktop.PowerManagement /org/freedesktop/PowerManagement
  org.freedesktop.PowerManagement.CanSuspendThenHibernate`.

Tested 2026-09-28 with `systemctl hibernate`: a 3.8GB image, powered off,
and the session resumed intact (uptime carried over, root `rw`). Right
after resume, one readahead READ failed on `sda` ("device offline or
changed") while the microSD reader was being reset. That's harmless:
readahead failures are dropped, and ext4 logged no errors and stayed
`clean`. It's the same reader re-attach that breaks S3, but a hibernate
resume is a cold boot and the reader was back in time.

## Networking

- **Realtek USB Ethernet (`enx00e04c68003e`, r8152-family 2.5G
  adapter):** the out-of-tree driver from
  <https://github.com/awesometic/realtek-r8152-dkms> (`~/github/realtek-r8152-dkms`)
  was once installed with DKMS, together with unloading `cdc_ether` /
  `cdc_ncm` / `cdc_mbim` and testing with `iperf3` against `pet630`,
  MTU 9000 and `ethtool -K` offloads. It was **later removed**
  (`sudo ./dkms-remove.sh`) because it failed to build against the 7.x
  kernels (confirmed by matt, 2026-09-27). `dkms status` is empty now, so the adapter gets the in-kernel
  driver. The MTU and offload tweaks were runtime-only and aren't
  persisted anywhere.
- **Tailscale** is installed (repo in `tailscale.list`) but
  `tailscaled` is **disabled** (`systemctl disable tailscaled`, around
  the time the agent account was created).

## Firmware and access

- MrChromebox firmware utility run twice
  (`curl -LOf https://mrchromebox.tech/firmware-util.sh && sudo bash
  firmware-util.sh`); the second run was the 2026-09-03 update to
  `MrChromebox-2606.1` (see `CLAUDE.md`).
- `agent-user` created with
  `create-agent-user.sh` from
  `https://gist.ergosteur.com/ergosteur/41df939a435741bfac431ac3d4b888d0`
  (a copy is in `~matt`).

## Software and third-party apt repos

Not hardware-specific, but needed to rebuild the machine. The
repositories in `/etc/apt/sources.list.d/` beyond neon's own:

| Source file | What | Set up via |
|---|---|---|
| `vscodium.sources` | VSCodium (`codium`) | paulcarroty key into `/usr/share/keyrings/vscodium-archive-keyring.gpg` |
| `google-chrome.sources` | Google Chrome | installing the `.deb` from Downloads; set as `x-www-browser` / `gnome-www-browser` via `update-alternatives` |
| `docker.sources` | Docker CE | Docker's official apt instructions; `matt` added to the `docker` group |
| `ansible-ubuntu-ansible-noble.sources` | `ansible-core` | `ppa:ansible/ansible` |
| `nodesource.sources` | Node.js 24 | `deb.nodesource.com/setup_24.x`; then `npm i -g @google/gemini-cli` |
| `tailscale.list` | Tailscale | (disabled, see above) |

Other tools installed by hand:
- **`bws`** (Bitwarden Secrets Manager CLI) v2.0.0: unzipped into
  `/opt/bitwarden/bws/`, symlinked to `/usr/local/bin/bws`, with
  completions in `/opt/bitwarden/bws/completions.bash` sourced from
  `~/.bashrc`.
- **GlobalProtect VPN** (`PanGPLinux-6.3.3-c31`, `gp_install.sh`, from
  `~/Downloads`).
- `pipx`: `mkdocs` plus the injected `mkdocs-material`.
- Claude Code (`curl -fsSL https://claude.ai/install.sh | bash`).

Notable apt packages installed on top of neon: `codium ffmpeg yt-dlp
imagemagick python3-pip python3-venv pipx python3-bs4 build-essential
ncdu bwm-ng wireshark mpv filezilla rpi-imager ntfs-3g exfatprogs
exfat-fuse hfsutils hfsprogs samba samba-vfs-modules krdc remmina ssh
iperf3 netcat-openbsd htop btop git pavucontrol-qt bash-completion
tesseract-ocr{,-eng,-fra,-kor} inxi lsscsi qrencode localepurge
zram-tools dkms`.

Git identities: `~/.gitconfig` plus `~/.gitconfig.ergosteur` and
`~/.gitconfig.tfo`, with SSH key aliases in `~/.ssh`:
`git_ssh_id_ergosteur` → `id_rsa`, and `git_ssh_id_myip4tfo` →
`id_ed25519`.

## Dead ends in the history (nothing left installed)

- `~/github/chell_audio` (iofq's chell audio repo) was cloned and
  `linux-modules-extra` / `snd_soc_skl` poked at, before switching to
  WeirdTreeThing's scripts (see `AUDIO.md`).
- `ntfs2btrfs` / `ntfsfix` against a `/dev/sda1` of the time. That was
  an external data card before this layout, not the current neon
  install, whose `sda1` is its ESP.