Your Debian 11 or 12 server keeps shutting down repeatedly throughout the day
Your Debian 11 or 12 server keeps shutting down repeatedly throughout the day
Hi, so I've been running a Debian Server at home since 2022. It was running stable until 4 months ago. Since then it crashes once or even twice a week. I mean it really crashes hard to the point it has to reboot itself. The logs don't show anything. Memtest came back with no errors, so the RAM seems fine. I am going to switch out the powersupply next. Here some specs about the server: Mainboard: MSI H510I PRO WIFI Mainboard Mini-ITX CPU: Intel Core i5-11600 RAM 64GB: Mushkin Wave - D464GB 2800-17 Redline FB G3 K2 MSK PSU: Seasonic Prime Fanless PX-500 HDD: 2 x Toshiba MG09ACA18TE SSD: Samsung SSD 980 PRO 1TB NIC: MellanoxConnectX-3 Cooling (CPU): be quiet! Dark Rock TopFlow 2 BK031 Fans: 4 x Noctua (2x 80mm + 2x140mm) In my next attempt to narrow down the cause I will install a new PSU: Seasonic Prime 650W Ultra 80 Plus Titanium Full Modular On my server I am running some apllications as well all in docker containers: Jellyfin AMP (for game servers) Nginx-Proxy-Manager There is also a cronjob running every minute to update my DNS-Entry in case my IP changes since I don't have a fixed IP address. If you need further information, logs or else please just ask. Syslog from the latest crash: 2024-04-24T09:59:03.256792+02:00 jensi5server systemd[1]: Starting amptasks.service - AMP Instance Manager Pending Tasks... 2024-04-24T09:59:03.329635+02:00 jensi5server ampinstmgr[224751]: [Info] AMP Instance Manager v2.5.0.6 built 17/04/2024 22:10 2024-04-24T09:59:03.337685+02:00 jensi5server ampinstmgr[224751]: [Info] Stream: Mainline / Release - built by CUBECODERS/buildbot on CCL-DEV 2024-04-24T09:59:03.641161+02:00 jensi5server systemd[1]: amptasks.service: Deactivated successfully. 2024-04-24T09:59:03.641294+02:00 jensi5server systemd[1]: Finished amptasks.service - AMP Instance Manager Pending Tasks. 2024-04-24T10:00:04.772521+02:00 jensi5server systemd-modules-load[373]: Inserted module 'coretemp' 2024-04-24T10:00:04.772593+02:00 jensi5server kernel: [ 0.000000] Linux version 6.1.0-20-amd64 ([email protected]) (gcc-12 (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Deb> 2024-04-24T10:00:04.772608+02:00 jensi5server kernel: [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-6.1.0-20-amd64 root=/dev/mapper/system-root ro quiet crashkernel=256M crashkernel=384M> 2024-04-24T10:00:04.772609+02:00 jensi5server kernel: [ 0.000000] BIOS-provided physical RAM map: 2024-04-24T10:00:04.772597+02:00 jensi5server systemd[1]: [email protected]: Deactivated successfully. 2024-04-24T10:00:04.772609+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009dfff] usable 2024-04-24T10:00:04.772616+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x000000000009e000-0x000000000009efff] reserved 2024-04-24T10:00:04.772616+02:00 jensi5server systemd[1]: Finished [email protected] - Load Kernel Module drm. Here the kern.log: 2024-04-24T10:00:04.772593+02:00 jensi5server kernel: [ 0.000000] Linux version 6.1.0-20-amd64 ([email protected]) (gcc-12 (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Deb> 2024-04-24T10:00:04.772608+02:00 jensi5server kernel: [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-6.1.0-20-amd64 root=/dev/mapper/system-root ro quiet crashkernel=256M crashkernel=384M> 2024-04-24T10:00:04.772609+02:00 jensi5server kernel: [ 0.000000] BIOS-provided physical RAM map: 2024-04-24T10:00:04.772609+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009dfff] usable 2024-04-24T10:00:04.772616+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x000000000009e000-0x000000000009efff] reserved 2024-04-24T10:00:04.772617+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x000000000009f000-0x000000000009ffff] usable 2024-04-24T10:00:04.772620+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x00000000000a0000-0x00000000000fffff] reserved 2024-04-24T10:00:04.772623+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000849fefff] usable 2024-04-24T10:00:04.772623+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x00000000849ff000-0x0000000084ffefff] reserved 2024-04-24T10:00:04.772624+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x0000000084fff000-0x00000000850fefff] ACPI data 2024-04-24T10:00:04.772625+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x00000000850ff000-0x00000000852fefff] ACPI NVS 2024-04-24T10:00:04.772626+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x00000000852ff000-0x0000000085efefff] reserved 2024-04-24T10:00:04.772627+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x0000000085eff000-0x0000000085ffefff] type 20 2024-04-24T10:00:04.772629+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x0000000085fff000-0x0000000085ffffff] usable 2024-04-24T10:00:04.772631+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x0000000086000000-0x0000000089ffffff] reserved 2024-04-24T10:00:04.772632+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x000000008a600000-0x000000008a7fffff] reserved 2024-04-24T10:00:04.772633+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x000000008b000000-0x00000000907fffff] reserved 2024-04-24T10:00:04.772633+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x00000000e0000000-0x00000000efffffff] reserved 2024-04-24T10:00:04.772634+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x00000000fe000000-0x00000000fe010fff] reserved 2024-04-24T10:00:04.772634+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x00000000fec00000-0x00000000fec00fff] reserved 2024-04-24T10:00:04.772636+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x00000000fed00000-0x00000000fed00fff] reserved 2024-04-24T10:00:04.772637+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x00000000fed20000-0x00000000fed7ffff] reserved 2024-04-24T10:00:04.772638+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x00000000fee00000-0x00000000fee00fff] reserved 2024-04-24T10:00:04.772638+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved 2024-04-24T10:00:04.772639+02:00 jensi5server kernel: [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000106f7fffff] usable 2024-04-24T10:00:04.772639+02:00 jensi5server kernel: [ 0.000000] NX (Execute Disable) protection: active 2024-04-24T10:00:04.772641+02:00 jensi5server kernel: [ 0.000000] efi: EFI v2.70 by American Megatrends 2024-04-24T10:00:04.772642+02:00 jensi5server kernel: [ 0.000000] efi: ACPI=0x85269000 ACPI 2.0=0x85269014 TPMFinalLog=0x8526c000 SMBIOS=0x85d1d000 SMBIOS 3.0=0x85d1c000 MEMATTR=0x7f26a198 ES> 2024-04-24T10:00:04.772643+02:00 jensi5server kernel: [ 0.000000] secureboot: Secure boot disabled 2024-04-24T10:00:04.772643+02:00 jensi5server kernel: [ 0.000000] SMBIOS 3.3.0 present. 2024-04-24T10:00:04.772644+02:00 jensi5server kernel: [ 0.000000] DMI: Micro-Star International Co., Ltd. MS-7D16/H510I PRO WIFI (MS-7D16), BIOS 1.30 09/09/2021 2024-04-24T10:00:04.772644+02:00 jensi5server kernel: [ 0.000000] tsc: Detected 2800.000 MHz processor 2024-04-24T10:00:04.772646+02:00 jensi5server kernel: [ 0.000000] tsc: Detected 2808.000 MHz TSC 2024-04-24T10:00:04.772647+02:00 jensi5server kernel: [ 0.000615] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved 2024-04-24T10:00:04.772648+02:00 jensi5server kernel: [ 0.000618] e820: remove [mem 0x000a0000-0x000fffff] usable 2024-04-24T10:00:04.772648+02:00 jensi5server kernel: [ 0.000629] last_pfn = 0x106f800 max_arch_pfn = 0x400000000 2024-04-24T10:00:04.772648+02:00 jensi5server kernel: [ 0.000781] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT 2024-04-24T10:00:04.772649+02:00 jensi5server kernel: [ 0.001497] last_pfn = 0x86000 max_arch_pfn = 0x400000000 2024-04-24T10:00:04.772649+02:00 jensi5server kernel: [ 0.020600] esrt: Reserving ESRT space from 0x000000008009a418 to 0x000000008009a450. 2024-04-24T10:00:04.772653+02:00 jensi5server kernel: [ 0.020605] e820: update [mem 0x8009a000-0x8009afff] usable ==> reserved 2024-04-24T10:00:04.772653+02:00 jensi5server kernel: [ 0.020629] Using GB pages for direct mapping 2024-04-24T10:00:04.772654+02:00 jensi5server kernel: [ 0.020965] RAMDISK: [mem 0x34057000-0x36022fff] 2024-04-24T10:00:04.772654+02:00 jensi5server kernel: [ 0.020968] ACPI: Early table checksum verification disabled
I've noticed more crashes lately, the most recent one just a few minutes ago. All game servers were taken down before that. At the moment, only two programs are running: Jellyfin NGINX-Proxy-Manager and I plan to replace the PSU today. If you have any other ideas on where to check for errors, feel welcome to share. Otherwise, let me know if you need help with logs or anything else.
Updated the power supply settings. RAM usage tracking didn’t reveal any spikes over 4%, even with three crashes that day. Monitoring temperatures is also important now. Could there be another factor I’m overlooking? Please let me know if you have more ideas.
Up to now there hasn't been any new crash after I replaced the PSU. I looked inside the old one and saw no obvious damage, but I think I should test it with a multimeter under load. Perhaps because it was cooled passively. For now I'll keep this post open for a few days and update if anything changes. Here are some images from the interior of the old PSU.
Here’s a revised version of your message:
Hello @BoomerDutch, sorry for the delayed reply. I wasn’t very familiar with the @ command here. Now I’ve received an uptime of 1d, which sounds impressive—but I’m still puzzled about why the old PSU started showing issues only after two years. Before I mark this resolved, I’ll keep this thread open for about a week. If anything changes, I’ll update it.
Regarding the other logs, I hope this matches what you were looking for: journalctl: Apr 24 15:33:10 jensi5server systemd[1157] - Reached target exit.target - Exit the Session. -- Boot dc8883febe6348528a7e3ec455f1240a -- Apr 25 11:00:02 jensi5server systemd[72841] - Queued start job for default target default.target. Apr 25 11:00:02 jensi5server systemd[72841] - Created slice app.slice - User Application Slice. Apr 25 11:00:02 jensi5server systemd[72841] - Reached target paths.target - Paths. Apr 24 10:45:04 jensi5server systemd[26827] - Closed gpg-agent.socket - GnuPG cryptographic agent and passphrase cache. Apr 24 10:45:04 jensi5server systemd[26827] - Closed pk-debconf-helper.socket - debconf communication socket. Apr 24 10:45:04 jensi5server systemd[26827] - Removed slice app.slice - User Application Slice. Apr 24 10:45:04 jensi5server systemd[26827] - Reached target shutdown.target - Shutdown. Apr 24 10:45:04 jensi5server systemd[26827] - Finished systemd-exit.service - Exit the Session. Apr 24 15:13:29 jensi5server systemd[114376] - Queued start job for default target default.target. Apr 24 15:13:29 jensi5server systemd[114376] - Created slice app.slice - User Application Slice. Apr 24 15:13:29 jensi5server systemd[114376] - Reached target paths.target - Paths. Apr 24 15:13:29 jensi5server systemd[114376] - Reached target timers.target - Timers. Apr 24 15:13:29 jensi5server systemd[114376] - Starting dbus.socket - D-Bus User Message Bus Socket... Apr 24 15:13:29 jensi5server systemd[114376] - Listening on dirmngr.socket - GnuPG network certificate management daemon. Apr 24 15:13:29 jensi5server systemd[114376] - Listening on gpg-agent-browser.socket - GnuPG cryptographic agent and passphrase cache (accessible for web browsers). Apr 24 15:13:29 jensi5server systemd[114376] - Listening on gpg-agent-extra.socket - GnuPG cryptographic agent and passphrase cache (restricted). Apr 24 15:13:29 jensi5server systemd[114376] - Listening on gpg-agent-ssh.socket - GnuPG cryptographic agent (ssh-agent emulation). Apr 24 15:13:29 jensi5server systemd[114376] - Listening on gpg-agent.socket - GnuPG cryptographic agent and passphrase cache. Apr 24 15:13:29 jensi5server systemd[114376]: Listening on pk-debconf-helper.socket - debconf communication socket. Apr 24 15:13:29 jensi5server systemd[114376]: Listening on dbus.socket - D-Bus User Message Bus Socket. Apr 24 15:13:29 jensi5server systemd[114376]: Reached target sockets.target - Sockets. Apr 24 15:13:29 jensi5server systemd[114376] - Reached target basic.target - Basic System. Apr 24 15:13:29 jensi5server systemd[114376] - Reached target default.target - Main User Target. Startup completed in 106ms.
A few clarifications:
- I kept the original tone but made slight adjustments for clarity and flow.
- Numbers and timestamps were preserved where relevant.
- Technical details were rephrased to enhance readability.