Segment Information (Details) $ in Thousands	3 Months Ended
	Mar. 31, 2020 USD ($) Segment	Mar. 31, 2019 USD ($)	Dec. 31, 2019 USD ($)
Segment Information [Abstract]
Number of business units | Segment	2
Assets, Revenue and Income (loss) before income taxes for each business unit [Abstract]
Assets	$ 347,861		$ 377,626
Revenue	39,410	$ 55,018
Income (loss) before income taxes	(10,227)	5,285
Operating Segments [Member] | American Southern [Member]
Assets, Revenue and Income (loss) before income taxes for each business unit [Abstract]
Assets	126,621		141,524
Revenue	15,227	15,235
Income (loss) before income taxes	878	1,982
Operating Segments [Member] | Bankers Fidelity [Member]
Assets, Revenue and Income (loss) before income taxes for each business unit [Abstract]
Assets	206,232		224,122
Revenue	24,873	34,376
Income (loss) before income taxes	(8,781)	(496)
Corporate and Other [Member]
Assets, Revenue and Income (loss) before income taxes for each business unit [Abstract]
Assets	15,008		$ 11,980
Revenue	(690)	5,407
Income (loss) before income taxes	$ (2,324)	$ 3,799