Core was generated by python3.9: middlewared (zettarepl)

Description

Core was generated by `python3.9: middlewared (zettarepl)'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 _sleepq_remove (sq=0x800aa42c0, td=0x8078af000) at /truenas-releng/freenas/_BE/os/lib/libthr/thread/thr_sleepq.c:153
153 SLIST_REMOVE_HEAD(&sq->sq_freeq, sq_flink);
[Current thread is 1 (LWP 102089)]

This is from an enterprise user system and they are routinely having zettarepl problems as well as core dumps so this is probably a good system to try and figure out what's going on.

Problem/Justification

None

Impact

None

Activity

Show:

Bug Clerk January 10, 2022 at 10:00 PM

Vladimir Vinogradenko January 10, 2022 at 9:51 PM

Reassigning to OS team — seems to be a crash in python or OS libraries.

Bug Clerk January 10, 2022 at 9:25 PM

Vladimir Vinogradenko January 10, 2022 at 9:20 PM

it crashes somewhere deep inside python or FreeBSD, I am not in the power to debug that. It is also not reproducible locally.

Limiting the number of simultaneously running replication seems a good idea. I can't imagine a case where running 15 replications in parallel would benefit performance. If the network is bandwidth-limited for a single connection, 5 parallel connections are usually enough to utilize its full capacity. The current default is "infinity", I propose to set it to 5 and consider this resolved.

Also I found out we are missing the UI for this option. I created an issue for 22.MM https://jira.ixsystems.com/browse/NAS-114241 should I target it to 13.0?

Bill O'Hanlon December 7, 2021 at 5:18 PM

We asked the customer to run the following command, per Vlad:
midclt call replication.config.update '{"max_parallel_replication_tasks": 1}'
 

That has prevented the crashes, so it is an adequate workaround in the short term.

Complete

Details

Assignee

Reporter

Labels

Time remaining

0m

Components

Affects versions

Priority

Katalon Platform

Created October 4, 2021 at 7:41 PM
Updated July 1, 2022 at 5:42 PM
Resolved January 10, 2022 at 10:15 PM

Flag notifications