Your baby is born with a sucking reflex, and when it comes to feeding this is what they’ll use for their first few months of life.
The other reflex your baby has is an extrusion reflex. This is when they'll push any food placed on their tongue out the front of their mouth.
This is completely normal, and doesn't mean they don't like their food - it just means they're not capable of swallowing it yet. Somewhere between four and six months the extrusion reflex should disappear, and it's from this point that you can look at giving your baby new types of food