fusion-mpt SAS problems on 12.1-RELEASE

Description

This is my SAS HBA, I've got SATA drives connected to it, and have been running it since 11.1.

mps0@pci0:2:0:0: class=0x010700 card=0x040015d9 chip=0x00721000 rev=0x03 hdr=0x00
vendor = 'Broadcom / LSI'
device = 'SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon]'
class = mass storage
subclass = SAS

I had catastrophic drive failure problems when I was on 12.1-U1. Going back to 12.0, everything is fine and stable.

I had constant this in dmesg:

mps0: IOC Fault 0x40000d04, Resetting
mps0: Reinitializing controller
mps0: Firmware: 20.00.04.00, Driver: 21.02.00.00-fbsd
mps0: IOCCapabilities: 1285c
(da9:mps0:0:17:0): Invalidating pack
da9 at mps0 bus 0 scbus0 target 17 lun 0
da9: s/n WD-WMC5C0D6YZ1L detached
mps0: Error reading device 0xd SATA PASSTHRU; iocstatus= 0x47
mps0: Error reading device 0xd SATA PASSTHRU; iocstatus= 0x804b
mps0: Sleeping 3 seconds after SATA ID error to wait for spinup

It would bounce across disks, it didn't matter what was actually going on. The system died hard, and reboots and other things would not fix it. I had to go back to 12.0-RELEASE.

If I boot up 12.1, everything starts freaking out, and i have data corruption problems, because drives get randomly removed, detached and attached. It was rather chaotic to try and figure out.

Problem/Justification

None

Impact

None

Linked work items

duplicates

NAS-108881

Panics on mps(4) firmware timeouts

SmartDraw Connector

Katalon Manual Tests (BETA)

Activity

David Kowis

January 25, 2021 at 3:30 AM

It's gonna take me a bit, because I only just got done with 4+ hours of data recovery

I'm not quite willing to go down that hole again on a Sunday night, when I've got work tomorrow. I can give it a shot next weekend tho.

Alexander Motin

January 25, 2021 at 2:31 AM

David, could you try to set hw.mps.max_io_pages loader tunable to 32 and check whether it solve your problem?

David Kowis

January 24, 2021 at 1:54 AM

I used the wrong numbers, it was on 12.0-U1.1 that everything died, not 12.1. Being on 12.0-RELEASE is still safe and stable, however.

David Kowis

January 24, 2021 at 1:52 AM

No, I have the latest firmware installed.

root@freenas[~]# dmesg | grep mps0
mps0: <Avago Technologies (LSI) SAS2008> port 0x8000-0x80ff mem 0xdf600000-0xdf603fff,0xdf580000-0xdf5bffff irq 27 at device 0.0 numa-domain 0 on pci2
mps0: Firmware: 20.00.04.00, Driver: 21.02.00.00-fbsd
mps0: IOCCapabilities: 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>

David Kowis

January 24, 2021 at 1:41 AM

https://www.freebsd.org/releases/12.1R/relnotes.html

Looking at the release notes, the mps/mpr drivers got updated for "stability." I wonder why I had the exact opposite effect.

Firmware for the SAS controller hasn't been updated in years, so I don't think that I'm out of date there.

Resize work item view side panel

Duplicate

Details

Assignee

Triage Team

Reporter

David Kowis

Components

Fix versions

N/A

Affects versions

12.0-RELEASE

Priority

Low

More fields

Katalon Platform

Created January 24, 2021 at 1:25 AM

Updated July 1, 2022 at 5:14 PM

Resolved January 27, 2021 at 5:33 PM