When you decide to learn Russian, you're immediately greeted with a few interesting challenges right away on day one. You have to learn a brand new alphabet just to read and write in this new language!
Fortunately, it's really not hard to learn the Russian alphabet, in fact, I think it's easier and better than ours because there is only one sound per letter. But there are two characters in the Russian alphabet that will leave you perplexed for a long time as you take on the challenge of this new language: the soft sign (ь) and the hard sign (ъ).
They are not letters. These two signs are merely modifiers — they manage something called palatalization. (Yeah, I had to check the spelling three times for that!)
Palatalization in Russian
The word palatalization refers to the act of moving the sound to the palate — the center of the roof of your mouth, directly underneath the nasal cavity. When you do this to a sound, it is called "soft".
So for example, when you see the letter д, you pronounce it much the same as a D in English, pronouncing sound in the your throat and touching the tip of your tongue to the back of your teeth. But when the д is softened (дь) your tongue should curl back just a bit, so the tip touches the palate, and rather than producing the sound in your throat, it should be moved forward to the middle of the mouth. And the same goes, in theory, for all the rest of the consonants.
The description sounds complicated and difficult. The reality is actually quite simple: just smile a bit, and exhale a tiny little "ih" sound as you voice the consonant. That "ih" sound is the key to making it work. (And I wish someone had explained this to me properly when I was first learning.)
You see, when you learn the vowels, you are taught а is "aah", and я is "yaah", etc. But that's not really true. The soft vowels are actually just short-hand for the regular vowels plus a soft sign!
(In spite of being rather similar, the vowels и and ы are not compatible because of the characteristics of the mouth when they're formed. But they do correspond.)
The purpose of soft vowels is simple: by creating five new vowel characters which include the implied soft sign, you greatly reduct the number of times a soft sign has to appear in a word. Rather than writing зьэльоный you can write зелёный. Rather than writing тьотьа, you can write тётя. It's much more convenient!
But in practice, the two are basically the same. And that's the reason why the sounds you're hearing when Russians speak don't seem to match the sounds you were taught when you were learning Russian. When you see тётя, you should pronounce it as тьотьа. That is, rather than saying "tyo-tya" as you were taught, you should be saying "iht'o-iht'a".
Our instuctors and our books and CD lessons have failed us. Again. This is the reason I spent two years struggling with words like маленький and малинки, and why I sounded funny when I said the word очень or мальчик. It's also one of the reasons I'm not using any instructors or learning materials this year.
So why the hard sign?
Thanks to the soft vowels, there are a lot fewer soft signs being written. But now we have soft signs being implied everywhere. So what if you want to use that soft vowel, but you don't want it to soften the preceding consonant?
Simple: just use a hard sign to protect the consonant. This becomes very important with words like сесть and съесть, where the first word is the verb to sit, but the second is the perfective form of the verb to eat. The hard с prefix is used to modify a verb — you don't want that to get lost!
So in summary, all consonants are hard by default. They become palatalized, or soft, when followed by the soft sign (ь) or any soft vowel (е, ё, и, ю, я). But they remain hard if immediately followed by the hard sign (ъ).
Hopefully this explanation helps to clear up some confusion about soft sounds, and about Russian pronunciation in general. Listen for that brief little "ih" sound being produced when native Russian speakers utter a palatalized consonant, and you'll instantly understand and improve your own speech.