It might not be obvious right away but chambers are colored as follows: A red altar means that a chamber has not yet been explored, it is currently available to enter. A green altar means you’ve completed the chamber and have claimed a reward for finishing it.