Numbers 20:7-12 (NKJV)
Then the Lord spoke to Moses, saying, “Take the rod; you and your brother Aaron gather the congregation together. Speak to the rock before their eyes, and it will yield its water; thus you shall bring water for them out of the rock, and give drink to the congregation and their animals.” So Moses took the rod from before the Lord as He commanded him.
And Moses and Aaron gathered the assembly together before the rock; and he said to them, “Hear now, you rebels! Must we bring water for you out of this rock?” Then Moses lifted his hand and struck the rock twice with his rod; and water came out abundantly, and the congregation and their animals drank.
Then the Lord spoke to Moses and Aaron, “Because you did not believe Me, to hallow Me in the eyes of the children of Israel, therefore you shall not bring this assembly into the land which I have given them.”
So what did Moses do wrong? He smote the rock instead of speaking to it. Minor detail right? Well, God is a God of details. Just look at the intricacy of his Creation. The devil is into doing things almost the right way but changing a few of the details. That's where Moses went wrong. Furthermore Moses did not represent the grace of God. He made it appear as if God is saying "Alright already, enough complaining, here is your stinking water." That is not a very good representation of God's gift and miracle of providence is it? So God was gracious but he was angry with Moses too because Moses did not do exactly as God had said.
All this makes sense but I don't think that it is the worst of what Moses did at this point. Notice how he spoke to the people: "Must we bring water out of this rock?" What's with the we business. Is Moses going to do the miracle or is God going to do it? It is saying "Must God and I do this for you." What arrogance and pride on Moses part. But aren't you and I guilty of this anytime we don't give God the complete credit for our good works? Or worse, perhaps we think that we did something meritorious by exercising faith in Jesus Christ and were born again as a result.
Ephesians 2:8-10 (NKJV)
For by grace you have been saved through faith, and that not of yourselves; it is the gift of God, not of works, lest anyone should boast. For we are His workmanship, created in Christ Jesus for good works, which God prepared beforehand that we should walk in them.