Just because we don't necessarily remember being babies doesn't mean that it's not when we did a lot of formative learning. Experts will generally agree that babies are "taking it all in" so ignoring teaching moments with the little tyke is not a wise choice.
Surprisingly, many parents are choosing to start "teaching" while the little tyke is still an infant by reading to the baby. Will she be able to register what your words are saying? There is nothing proving that, but that calm bonding time with the infant is very important, so why not start a library of picture books a little earlier than you might have otherwise done that?
If you are thinking something like "if it's just hearing a voice that's important, why buy picture books so early, why not just read the newspaper or the month's utility bills to the baby?" The answer would be that most books written specifically for children have that certain "sing-songy" rhythm that is pleasant for us to read and for children to hear. In a very short time, the baby will be old enough to see the pictures and that is when you can point out what certain objects, shapes, colors, etc. are.
Having plenty of books available all throughout a child's life is very important, but there are other vital things to expose the baby to as well. A collection of many different types of music is one of those things. Have a CD of classical music playing when you are having quiet time with the baby or some perky children's songs when it is play time.
If you make sure that music is a part of the baby's life from the very beginning, don't be surprised to hear him or her "singing along" by making different baby sounds than you are used to hearing. One time when it's usually not good (for various reasons) to play music is for when you are trying to have the baby go to sleep. Babies constantly process everything around them, so that would just be a distraction and one more thing to process when she should be going to sleep. I have also heard that some babies, if they are used to being put to sleep with music, will eventually come to a point where falling to sleep *without* the music becomes quite a problem.
Don't just let the music part of "very early childhood education" be about CDs or videos, either. Allow your son or daughter to hear you singing from their earliest memory. It would be a good idea to ask older siblings to sing with the baby when they can, too.
One good thing to remember although it isn't actually "teaching" per se, is that babies can pick up on voice tones. Speak softly with gentle tones because they pick up on when someone is angry or upset. It isn't really a "rule" just for spending time around babies; it is a good thing to remember in life no matter with whom we are dealing.
Keeping your voice at a nice gentle level at all times isn't the only thing about "speech" to teach baby. Talk to the little bundle of love every chance you get. Don't let the thought that he won't understand what you are saying stop you from doing that. Carry him around with you throughout the day if you are able and spend some quality time with him. He may not understand and he most likely will not remember it, but then again, a lot of study still needs to be done about those formative years.
You won't be playing games for learning until the baby is older, but it is a good idea to play counting games with baby when he or she is still a toddler. I don't mean infants for this one, but "one, two, three books!" or "red ball" takes only a moment to say to older babies or toddlers and even if they are just beginning to talk, you may be surprised to hear a repeated "one, two, three!" for some other object in the house.
There will be many chances to teach baby things when she's a toddler and preschooler, but don't overlook the first months of life. The whole world is an open book to the little prince or princess.