It’s nice to have luxuries, but before you can have any of that, your people need clothes on their backs and food on their tables. Only increase the luxury goods you bring in once you’ve got your economy and trade routes running.