Early on you may have a difficult time upgrading weapons due to lack of Smithing Stones. The best place to find them early on is at mines and merchants. After revealing areas of the map with Map Fragments, you can spot a mine with a cave-like illustration (see image). These mines usually contain Smithing Stones for you to upgrade armaments/weapons at the Smithing Table.