The SD card is a key part of the Raspberry Pi; it provides the initial storage for the Operating System and files. Storage can be extended through many types of USB connected peripherals.
When the Raspberry Pi is 'switched on', i.e. connected to a power supply, a special piece of code called the bootloader is executed, which reads more special code from the SD card that is used to start up the Raspberry Pi. If there is no SD card inserted, it will not start. Do NOT push in or pull out an SD card while the Raspberry Pi is connected to the power, as this is likely to corrupt the SD.The SD card must be formatted, or written to, in a special way that means the Raspberry Pi can read the data it needs to start properly. If you are new to this check the instructions, or buy a pre-formatted SD card.
One advantage to using an SD card like this is that you can have several SD cards, each with a different operating system, or a different purpose. Simply power off, switch cards, and reconnect the power. You have a different computer to play with.