Linux Kernel 6.7.arch3.1 breaks mssql-server (AUR)

topslick · January 15, 2024, 9:07pm

Hi. I’ve got kernel 6.7.arch3.1 and (luckily) 6.6.11-2 installed. When booting 6.6.11-2, the sql server starts correctly. However, when booting 6.7.arch3.1, the sql server dumps and refuses to start. So if you depend on mssql-server, be careful when upgrading. systemctl and journalctl for the 6.7.arch3.1 boot and systemctl for the 6.6.11-2 below.

If this is not the right category or wording, sorry - just wanted to signal the topic for users depending on this (not always our choice). All packages are the latest stable, as far as i know.

** fail on kernel 6.7.arch3.1: systemctl status**
jan 15 20:30:37 sqlservr[959]: Successfully captured dump: /var/opt/mssql/log/core.sqlservr.1_15_2024_20_30_36.2808
jan 15 20:30:37 sqlservr[959]: Executing: /opt/mssql/bin/handle-crash.sh with parameters
jan 15 20:30:37 sqlservr[959]: /opt/mssql/bin/sqlservr
jan 15 20:30:37 sqlservr[959]: /opt/mssql/bin
jan 15 20:30:37 sqlservr[959]: /var/opt/mssql/log/
jan 15 20:30:37 sqlservr[959]: /var/opt/mssql/log/core.sqlservr.1_15_2024_20_30_36.2808
jan 15 20:30:38 sqlservr[959]: Capturing core dump and information to /var/opt/mssql/log…
jan 15 20:30:38 sqlservr[959]: /opt/mssql/bin/crash-support-functions.sh: line 536: hash: dpkg: not found
jan 15 20:30:40 sqlservr[959]: Dump already generated: /var/opt/mssql/log/core.sqlservr.1_15_2024_20_30_36.2808, moving to /var/opt/mssql/log/core.sqlservr.2808.temp/core.sqlservr.2808.gdmp
jan 15 20:30:40 sqlservr[959]: Moving logs to /var/opt/mssql/log/core.sqlservr.2808.temp/log/paldumper-debug.log
jan 15 20:32:33 sqlservr[959]: /var/opt/mssql/log/core.sqlservr.01_15_2024_20_30_38.2808.tbz2
jan 15 20:32:33 systemd[1]: mssql-server.service: Main process exited, code=exited, status=1/FAILURE
jan 15 20:32:33 systemd[1]: mssql-server.service: Failed with result ‘exit-code’.
jan 15 20:32:33 systemd[1]: mssql-server.service: Unit process 4419 (handle-crash.sh) remains running after unit stopped.
jan 15 20:32:33 systemd[1]: mssql-server.service: Unit process 2808 (sqlservr) remains running after unit stopped.
jan 15 20:32:33 systemd[1]: mssql-server.service: Consumed 6.653s CPU time, 2.2G memory peak, 0B memory swap peak.
jan 15 20:32:33 systemd[1]: mssql-server.service: Scheduled restart job, restart counter is at 1.
jan 15 20:32:33 systemd[1]: mssql-server.service: Found left-over process 4419 (sh) in control group while starting unit. Ignoring.
jan 15 20:32:33 systemd[1]: mssql-server.service: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
jan 15 20:32:33 systemd[1]: mssql-server.service: Found left-over process 4420 (tar) in control group while starting unit. Ignoring.
jan 15 20:32:33 systemd[1]: mssql-server.service: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
jan 15 20:32:33 systemd[1]: mssql-server.service: Found left-over process 4422 (bzip2) in control group while starting unit. Ignoring.
jan 15 20:32:33 systemd[1]: mssql-server.service: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
jan 15 20:32:35 sqlservr[4425]: 2024-01-15 20:32:35.40 Server Setup step is FORCE copying system data file ‘C:\templatedata\model_replicatedmaster.mdf’ to '/var/opt/mssql/data/model_replicatedmaste>
jan 15 20:32:35 sqlservr[4425]: [178B blob data]
jan 15 20:32:35 sqlservr[4425]: [162B blob data]
jan 15 20:32:35 sqlservr[4425]: [160B blob data]
jan 15 20:32:37 sqlservr[4425]: [102B blob data]
jan 15 20:32:37 sqlservr[4425]: [43B blob data]
jan 15 20:32:37 sqlservr[4425]: [44B blob data]
jan 15 20:32:37 sqlservr[4425]: [40B blob data]
jan 15 20:32:37 sqlservr[4425]: [152B blob data]
jan 15 20:32:38 sqlservr[4423]: Successfully captured dump: /var/opt/mssql/log/core.sqlservr.1_15_2024_20_32_38.4425
jan 15 20:32:38 sqlservr[4423]: Executing: /opt/mssql/bin/handle-crash.sh with parameters
jan 15 20:32:38 sqlservr[4423]: /opt/mssql/bin/sqlservr
jan 15 20:32:38 sqlservr[4423]: /opt/mssql/bin
jan 15 20:32:38 sqlservr[4423]: /var/opt/mssql/log/
jan 15 20:32:38 sqlservr[4423]: /var/opt/mssql/log/core.sqlservr.1_15_2024_20_32_38.4425
jan 15 20:32:38 sqlservr[4423]: Capturing core dump and information to /var/opt/mssql/log…
jan 15 20:32:38 sqlservr[4423]: /opt/mssql/bin/crash-support-functions.sh: line 536: hash: dpkg: not found
jan 15 20:32:39 sqlservr[4423]: Dump already generated: /var/opt/mssql/log/core.sqlservr.1_15_2024_20_32_38.4425, moving to /var/opt/mssql/log/core.sqlservr.4425.temp/core.sqlservr.4425.gdmp
jan 15 20:32:39 sqlservr[4423]: Moving logs to /var/opt/mssql/log/core.sqlservr.4425.temp/log/paldumper-debug.log
jan 15 20:32:39 sqlservr[4423]: /var/opt/mssql/log/core.sqlservr.01_15_2024_20_32_38.4425.tbz2
jan 15 20:32:39 systemd[1]: mssql-server.service: Main process exited, code=exited, status=1/FAILURE

fail on kernel 6.7.arch3.1: journalctl
jan 15 20:30:37 sqlservr[959]: Successfully captured dump: /var/opt/mssql/log/core.sqlservr.1_15_2024_20_30_36.2808
jan 15 20:30:37 sqlservr[959]: Executing: /opt/mssql/bin/handle-crash.sh with parameters
jan 15 20:30:37 sqlservr[959]: /opt/mssql/bin/sqlservr
jan 15 20:30:37 sqlservr[959]: /opt/mssql/bin
jan 15 20:30:37 sqlservr[959]: /var/opt/mssql/log/
jan 15 20:30:37 sqlservr[959]: /var/opt/mssql/log/core.sqlservr.1_15_2024_20_30_36.2808
jan 15 20:30:38 sqlservr[959]: Capturing core dump and information to /var/opt/mssql/log…
jan 15 20:30:38 sqlservr[959]: /opt/mssql/bin/crash-support-functions.sh: line 536: hash: dpkg: not found
jan 15 20:30:40 sqlservr[959]: Dump already generated: /var/opt/mssql/log/core.sqlservr.1_15_2024_20_30_36.2808, moving to /var/opt/mssql/log/core.sqlservr.2808.temp/core.sqlservr.2808.gdmp
jan 15 20:30:40 sqlservr[959]: Moving logs to /var/opt/mssql/log/core.sqlservr.2808.temp/log/paldumper-debug.log
jan 15 20:32:33 sqlservr[959]: /var/opt/mssql/log/core.sqlservr.01_15_2024_20_30_38.2808.tbz2
jan 15 20:32:33 systemd[1]: mssql-server.service: Main process exited, code=exited, status=1/FAILURE
jan 15 20:32:33 systemd[1]: mssql-server.service: Failed with result ‘exit-code’.
jan 15 20:32:33 systemd[1]: mssql-server.service: Unit process 4419 (handle-crash.sh) remains running after unit stopped.
jan 15 20:32:33 systemd[1]: mssql-server.service: Unit process 2808 (sqlservr) remains running after unit stopped.
jan 15 20:32:33 systemd[1]: mssql-server.service: Consumed 6.653s CPU time, 2.2G memory peak, 0B memory swap peak.
jan 15 20:32:33 systemd[1]: mssql-server.service: Scheduled restart job, restart counter is at 1.
jan 15 20:32:33 systemd[1]: mssql-server.service: Found left-over process 4419 (sh) in control group while starting unit. Ignoring.
jan 15 20:32:33 systemd[1]: mssql-server.service: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
jan 15 20:32:33 systemd[1]: mssql-server.service: Found left-over process 4420 (tar) in control group while starting unit. Ignoring.
jan 15 20:32:33 systemd[1]: mssql-server.service: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
jan 15 20:32:33 systemd[1]: mssql-server.service: Found left-over process 4422 (bzip2) in control group while starting unit. Ignoring.
jan 15 20:32:33 systemd[1]: mssql-server.service: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
jan 15 20:32:35 sqlservr[4425]: 2024-01-15 20:32:35.40 Server Setup step is FORCE copying system data file ‘C:\templatedata\model_replicatedmaster.mdf’ to ‘/var/opt/mssql/data/model_replicatedmaster.mdf’.
jan 15 20:32:35 sqlservr[4425]: [178B blob data]
jan 15 20:32:35 sqlservr[4425]: [162B blob data]
jan 15 20:32:35 sqlservr[4425]: [160B blob data]
jan 15 20:32:37 sqlservr[4425]: [102B blob data]
jan 15 20:32:37 sqlservr[4425]: [43B blob data]
jan 15 20:32:37 sqlservr[4425]: [44B blob data]
jan 15 20:32:37 sqlservr[4425]: [40B blob data]
jan 15 20:32:37 sqlservr[4425]: [152B blob data]

success on kernel 6.6.11-2 : systemctl status
● mssql-server.service - Microsoft SQL Server Database Engine
Loaded: loaded (/usr/lib/systemd/system/mssql-server.service; enabled; pre> Active: active (running) since Mon 2024-01-15 21:04:51 CET; 11min ago
Docs: https://docs.microsoft.com/en-us/sql/linux
Main PID: 956 (sqlservr)
Tasks: 288
Memory: 2.4G (peak: 2.4G)
CPU: 34.328s
CGroup: /system.slice/mssql-server.service
├─ 956 /opt/mssql/bin/sqlservr
└─2798 /opt/mssql/bin/sqlservr

jan 15 21:07:23 sqlservr[2798]: [59B blob data]
jan 15 21:07:23 sqlservr[2798]: [58B blob data]
jan 15 21:07:23 sqlservr[2798]: [56B blob data]
jan 15 21:07:23 sqlservr[2798]: [57B blob data]
jan 15 21:07:23 sqlservr[2798]: [53B blob data]
jan 15 21:07:23 sqlservr[2798]: [124B blob data]
jan 15 21:10:45 sqlservr[2798]: [76B blob data]
jan 15 21:10:45 sqlservr[2798]: [75B blob data]
jan 15 21:12:41 sqlservr[2798]: [156B blob data]
jan 15 21:12:42 sqlservr[2798]: [195B blob data]

wai · January 20, 2024, 9:12pm

Same when I update kernel to version 6.7.1; rollback to 6.10.13 works for me.

Here is the message from syslog:

2024-01-20T12:07:44.714904-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715010-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715034-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715056-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715075-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715096-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715113-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715131-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715160-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715181-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715201-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715222-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715244-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715266-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715294-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715314-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715336-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715356-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715376-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715396-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715420-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715439-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715459-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715481-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715501-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715518-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715534-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715551-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715567-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715591-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.715608-05:00 myserver sqlservr[34975]: 2024-01-20T12:07:44.768398-05:00 myserver sqlservr[34973]: 2024-01-20T12:07:46.003825-05:00 myserver sqlservr[34973]: 2024-01-20T12:07:46.003918-05:00 myserver sqlservr[34973]: 2024-01-20T12:07:46.003954-05:00 myserver sqlservr[34973]: 2024-01-20T12:07:46.003986-05:00 myserver sqlservr[34973]: 2024-01-20T12:07:46.004014-05:00 myserver sqlservr[34973]: 2024-01-20T12:07:46.004045-05:00 myserver sqlservr[34973]: 2024-01-20T12:07:46.004069-05:00 myserver sqlservr[34973]: 2024-01-20T12:07:46.004094-05:00 myserver sqlservr[34973]: 2024-01-20T12:07:46.004123-05:00 myserver sqlservr[34973]: 2024-01-20T12:07:46.004148-05:00 myserver sqlservr[34973]: 2024-01-20T12:07:46.004177-05:00 myserver sqlservr[34973]: 2024-01-20T12:07:46.004201-05:00 myserver sqlservr[34973]: 2024-01-20T12:07:46.018508-05:00 myserver sqlservr[34973]: 2024-01-20T12:07:46.022448-05:00 myserver sqlservr[34973]: 2024-01-20T12:07:46.882780-05:00 myserver sqlservr[35733]: This program has encountered a fatal error and cannot continue running at Sat Jan 20 12:07:44 2024
The following diagnostic information is available:
Reason: 0x00000001
Signal: SIGABRT - Aborted (6)
Stack:
IP Function
---------------- --------------------------------------
000055928b7ab5e1 std::__1::bad_function_call::~bad_function_call()+0x92741
000055928b7aafa6 std::__1::bad_function_call::~bad_function_call()+0x92106
000055928b7aa52f std::__1::bad_function_call::~bad_function_call()+0x9168f
00007fd72ed54510 __sigaction+0x40
00007fd72eda20fc pthread_key_delete+0x14c
00007fd72ed54472 gsignal+0x12
00007fd72ed3e4b2 abort+0xd3
000055928b726f16 std::__1::bad_function_call::~bad_function_call()+0xe076
000055928b7b4864 std::__1::bad_function_call::~bad_function_call()+0x9b9c4
000055928b7e6b78 std::__1::bad_function_call::~bad_function_call()+0xcdcd8
000055928b7e695a std::__1::bad_function_call::~bad_function_call()+0xcdaba
000055928b72d38a std::__1::bad_function_call::~bad_function_call()+0x144ea
000055928b72d000 std::__1::bad_function_call::~bad_function_call()+0x14160
Process: 34975 - sqlservr
Thread: 35110 (application thread 0x230)
Instance Id: d8ab2ea6-f017-4708-a86d-aea1bfb28c67
Crash Id: 124e87e3-d8a8-46e8-9ff1-e37b73db1706
Build stamp: d6cdc92a98f2b00db04096ba71e3cf3002529f372e433b0a165fd4bdf3c75ee9
Distribution: Debian GNU/Linux trixie/sid
Processors: 24
Total Memory: 67333681152 bytes
Timestamp: Sat Jan 20 12:07:44 2024
Last errno: 2
Last errno text: No such file or directory
Capturing a dump of 34975
Successfully captured dump: /var/opt/mssql/log/core.sqlservr.1_20_2024_12_7_44.34975
Executing: /opt/mssql/bin/handle-crash.sh with parameters
handle-crash.sh
/opt/mssql/bin/sqlservr
34975
/opt/mssql/bin
/var/opt/mssql/log/

d8ab2ea6-f017-4708-a86d-aea1bfb28c67
124e87e3-d8a8-46e8-9ff1-e37b73db1706

/var/opt/mssql/log/core.sqlservr.1_20_2024_12_7_44.34975
Debian GNU/Linux trixie/sid
Capturing core dump and information to /var/opt/mssql/log…
dmesg: read kernel buffer failed: Operation not permitted

dimacat.invisible · February 20, 2024, 9:31am

Thank you. I have faced the same problem and try my best to fix it and I dunno what coz the problem. Revert back to 6.6.10 really fix it. Thanks man.

gipo355 · February 26, 2024, 10:00am

yeah confirming, mssql docker doesn’t work either

topslick · March 20, 2024, 8:12pm

upgraded mssql to 16.0.4115.5-1
upgraded kernel to 6.8.1-arch1-1
system starts correctly with the mssql-server.service up and running.
solved by upstream - thanks!

Cphusion · March 20, 2024, 9:36pm

It’s better to run such applications in a container, Microsoft has an official image for it.
https://hub.docker.com/_/microsoft-mssql-server