If you're not backstabbing an enemy properly it might be a few issues such as not being close enough or to the side of them. You might also have alerted them of your presence. Inflict critical damage more consistently by locking-on to your target while crouching. Get extra close and directly behind the enemy, you can even touch them before striking! Use normal attack (RB/R1) to assassinate.