Hart's potential departure fuels special election speculation