Minimum CPU features? || SIGILL: illegal instruction

[DDzwiedziu]

What are the minimum CPU requirements for models?

I have found https://localai.io/model-compatibility/, but there no information on specific CPU features/instructions.

As in the meantime I'm getting such errors when I try to generate a sample response from curl:

% docker container logs -t --details localai_api_1 |& grep -A 6 SIGIL
2024-10-20T13:50:29.795064133Z  1:50PM DBG GRPC(Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf-127.0.0.1:40209): stderr SIGILL: illegal instruction
2024-10-20T13:50:29.795086435Z  1:50PM DBG GRPC(Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf-127.0.0.1:40209): stderr PC=0x89101a m=5 sigcode=2
2024-10-20T13:50:29.795138093Z  1:50PM DBG GRPC(Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf-127.0.0.1:40209): stderr signal arrived during cgo execution
2024-10-20T13:50:29.795163767Z  1:50PM DBG GRPC(Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf-127.0.0.1:40209): stderr instruction bytes: 0xc4 0xe2 0x79 0x13 0xc9 0xc5 0xf2 0x59 0x15 0x49 0xc1 0x1f 0x0 0xc4 0x81 0x7a
2024-10-20T13:50:29.795182954Z  1:50PM DBG GRPC(Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf-127.0.0.1:40209): stderr 
2024-10-20T13:50:29.795192478Z  1:50PM DBG GRPC(Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf-127.0.0.1:40209): stderr goroutine 53 gp=0xc000104380 m=5 mp=0xc000100008 [syscall]:
2024-10-20T13:50:29.795230915Z  1:50PM DBG GRPC(Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf-127.0.0.1:40209): stderr runtime.cgocall(0x824c60, 0xc0000d3630)
--
2024-10-20T13:59:30.679969215Z  1:59PM DBG GRPC(Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf-127.0.0.1:41265): stderr SIGILL: illegal instruction
2024-10-20T13:59:30.679983692Z  1:59PM DBG GRPC(Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf-127.0.0.1:41265): stderr PC=0x89101a m=7 sigcode=2
2024-10-20T13:59:30.680016906Z  1:59PM DBG GRPC(Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf-127.0.0.1:41265): stderr signal arrived during cgo execution
2024-10-20T13:59:30.680108777Z  1:59PM DBG GRPC(Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf-127.0.0.1:41265): stderr instruction bytes: 0xc4 0xe2 0x79 0x13 0xc9 0xc5 0xf2 0x59 0x15 0x49 0xc1 0x1f 0x0 0xc4 0x81 0x7a
2024-10-20T13:59:30.680122347Z  1:59PM DBG GRPC(Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf-127.0.0.1:41265): stderr 
2024-10-20T13:59:30.680133194Z  1:59PM DBG GRPC(Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf-127.0.0.1:41265): stderr goroutine 50 gp=0xc000232000 m=7 mp=0xc0001a2708 [syscall]:
2024-10-20T13:59:30.680198941Z  1:59PM DBG GRPC(Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf-127.0.0.1:41265): stderr runtime.cgocall(0x824c60, 0xc00023f630)
--
2024-10-20T14:17:56.571303109Z  2:17PM DBG GRPC(ggml-model-q4_0.bin-127.0.0.1:41253): stderr SIGILL: illegal instruction
2024-10-20T14:17:56.571349270Z  2:17PM DBG GRPC(ggml-model-q4_0.bin-127.0.0.1:41253): stderr PC=0x842862 m=3 sigcode=2
2024-10-20T14:17:56.571392919Z  2:17PM DBG GRPC(ggml-model-q4_0.bin-127.0.0.1:41253): stderr signal arrived during cgo execution
2024-10-20T14:17:56.571418840Z  2:17PM DBG GRPC(ggml-model-q4_0.bin-127.0.0.1:41253): stderr instruction bytes: 0xc4 0xe2 0x79 0x13 0xc9 0xc5 0xf2 0x59 0x15 0x99 0x97 0x1e 0x0 0xc5 0xfa 0x11
2024-10-20T14:17:56.571507147Z  2:17PM DBG GRPC(ggml-model-q4_0.bin-127.0.0.1:41253): stderr 
2024-10-20T14:17:56.571554080Z  2:17PM DBG GRPC(ggml-model-q4_0.bin-127.0.0.1:41253): stderr goroutine 19 gp=0xc0000d0fc0 m=3 mp=0xc00006ce08 [syscall]:
2024-10-20T14:17:56.571604405Z  2:17PM DBG GRPC(ggml-model-q4_0.bin-127.0.0.1:41253): stderr runtime.cgocall(0x817b70, 0xc0000d9830)
--
2024-10-20T14:20:10.361612040Z  2:20PM DBG GRPC(llama-3.2-1b-instruct-q4_k_m.gguf-127.0.0.1:46625): stderr SIGILL: illegal instruction
2024-10-20T14:20:10.361624913Z  2:20PM DBG GRPC(llama-3.2-1b-instruct-q4_k_m.gguf-127.0.0.1:46625): stderr PC=0x89101a m=5 sigcode=2
2024-10-20T14:20:10.361723183Z  2:20PM DBG GRPC(llama-3.2-1b-instruct-q4_k_m.gguf-127.0.0.1:46625): stderr signal arrived during cgo execution
2024-10-20T14:20:10.361745052Z  2:20PM DBG GRPC(llama-3.2-1b-instruct-q4_k_m.gguf-127.0.0.1:46625): stderr instruction bytes: 0xc4 0xe2 0x79 0x13 0xc9 0xc5 0xf2 0x59 0x15 0x49 0xc1 0x1f 0x0 0xc4 0x81 0x7a
2024-10-20T14:20:10.361753828Z  2:20PM DBG GRPC(llama-3.2-1b-instruct-q4_k_m.gguf-127.0.0.1:46625): stderr 
2024-10-20T14:20:10.361761718Z  2:20PM DBG GRPC(llama-3.2-1b-instruct-q4_k_m.gguf-127.0.0.1:46625): stderr goroutine 24 gp=0xc000007880 m=5 mp=0xc000100008 [syscall]:
2024-10-20T14:20:10.361800351Z  2:20PM DBG GRPC(llama-3.2-1b-instruct-q4_k_m.gguf-127.0.0.1:46625): stderr runtime.cgocall(0x824c60, 0xc000163630)
--
2024-10-20T14:26:11.793325733Z  2:26PM DBG GRPC(ggml-model-q4_0.bin-127.0.0.1:34957): stderr SIGILL: illegal instruction
2024-10-20T14:26:11.793354219Z  2:26PM DBG GRPC(ggml-model-q4_0.bin-127.0.0.1:34957): stderr PC=0x842862 m=0 sigcode=2
2024-10-20T14:26:11.793362876Z  2:26PM DBG GRPC(ggml-model-q4_0.bin-127.0.0.1:34957): stderr signal arrived during cgo execution
2024-10-20T14:26:11.793372747Z  2:26PM DBG GRPC(ggml-model-q4_0.bin-127.0.0.1:34957): stderr instruction bytes: 0xc4 0xe2 0x79 0x13 0xc9 0xc5 0xf2 0x59 0x15 0x99 0x97 0x1e 0x0 0xc5 0xfa 0x11
2024-10-20T14:26:11.793397459Z  2:26PM DBG GRPC(ggml-model-q4_0.bin-127.0.0.1:34957): stderr 
2024-10-20T14:26:11.793406836Z  2:26PM DBG GRPC(ggml-model-q4_0.bin-127.0.0.1:34957): stderr goroutine 53 gp=0xc000104540 m=0 mp=0xd62140 [syscall]:
2024-10-20T14:26:11.793425050Z  2:26PM DBG GRPC(ggml-model-q4_0.bin-127.0.0.1:34957): stderr runtime.cgocall(0x817b70, 0xc0000d9830)
--
2024-10-20T14:34:58.452107904Z  2:34PM DBG GRPC(Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf-127.0.0.1:37717): stderr SIGILL: illegal instruction
2024-10-20T14:34:58.452133095Z  2:34PM DBG GRPC(Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf-127.0.0.1:37717): stderr PC=0x89101a m=3 sigcode=2
2024-10-20T14:34:58.452191096Z  2:34PM DBG GRPC(Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf-127.0.0.1:37717): stderr signal arrived during cgo execution
2024-10-20T14:34:58.452206310Z  2:34PM DBG GRPC(Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf-127.0.0.1:37717): stderr instruction bytes: 0xc4 0xe2 0x79 0x13 0xc9 0xc5 0xf2 0x59 0x15 0x49 0xc1 0x1f 0x0 0xc4 0x81 0x7a
2024-10-20T14:34:58.452243942Z  2:34PM DBG GRPC(Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf-127.0.0.1:37717): stderr 
2024-10-20T14:34:58.452253943Z  2:34PM DBG GRPC(Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf-127.0.0.1:37717): stderr goroutine 50 gp=0xc000238000 m=3 mp=0xc00006ae08 [syscall]:
2024-10-20T14:34:58.452266647Z  2:34PM DBG GRPC(Hermes-2-Pro-Llama-3-8B-Q4_K_M.gguf-127.0.0.1:37717): stderr runtime.cgocall(0x824c60, 0xc0000d3630)

CPU:

% lscpu
Architecture:             x86_64
  CPU op-mode(s):         32-bit, 64-bit
  Address sizes:          36 bits physical, 48 bits virtual
  Byte Order:             Little Endian
CPU(s):                   8
  On-line CPU(s) list:    0-7
Vendor ID:                GenuineIntel
  Model name:             Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
    CPU family:           6
    Model:                42
    Thread(s) per core:   2
    Core(s) per socket:   4
    Socket(s):            1
    Stepping:             7
    CPU(s) scaling MHz:   59%
    CPU max MHz:          3800,0000
    CPU min MHz:          1600,0000
    BogoMIPS:             6785,36
    Flags:                fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc c
                          puid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi fl
                          expriority ept vpid xsaveopt dtherm ida arat pln pts flush_l1d
Virtualization features:  
  Virtualization:         VT-x
Caches (sum of all):      
  L1d:                    128 KiB (4 instances)
  L1i:                    128 KiB (4 instances)
  L2:                     1 MiB (4 instances)
  L3:                     8 MiB (1 instance)
NUMA:                     
  NUMA node(s):           1
  NUMA node0 CPU(s):      0-7
Vulnerabilities:          
  Gather data sampling:   Not affected
  Itlb multihit:          KVM: Mitigation: Split huge pages
  L1tf:                   Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable
  Mds:                    Vulnerable: Clear CPU buffers attempted, no microcode; SMT vulnerable
  Meltdown:               Mitigation; PTI
  Mmio stale data:        Unknown: No mitigations
  Reg file data sampling: Not affected
  Retbleed:               Not affected
  Spec rstack overflow:   Not affected
  Spec store bypass:      Mitigation; Speculative Store Bypass disabled via prctl
  Spectre v1:             Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:             Mitigation; Retpolines; IBPB conditional; IBRS_FW; STIBP conditional; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
  Srbds:                  Not affected
  Tsx async abort:        Not affected