I did it!

Do I need to go kill the ugly bird lady boss before continuing to not screw up the elder's questline, or can I come back later?