Around three in 10 parents admit making negative comments about maths in front of their children, according to a poll commissioned by the Welsh government.
Maths results are the worst of any core subject in Wales behind English, Welsh and science with two new maths GCSEs to be introduced.
The Welsh government is launching a new campaign to encourage parents to help children with the subject.
Education Minister Huw Lewis said positive attitudes at home were vital.
He said: "It's fair to say that maths is suffering from an image problem, and as today's poll demonstrates, there is still work to do to change the view in some quarters which is that maths isn't really important and that it doesn't really matter what we say to children about it.
"We understand the value of strong numeracy skills, for life and for employment.
"If everyone gets behind this campaign and spreads the message that what you say counts when it comes to maths, I'm sure the benefits will add up."
The Welsh government-commissioned poll of 1,000 people in Wales carried out between 10 and 25 February made the following findings:
• Eighty per cent claimed to be either "good" or "ok" at maths, but 29% acknowledged making negative comments about the subject in front of their children
• Respondents admitted to saying statements like: "Maths was never my strong point" or "Don't ask me to do maths, I'm rubbish at it"
• Forty six per cent said they were good at maths and 40% said they were ok at it
• Eighty eight per cent said they used maths at least weekly to check bank statements, 79% to plan a weekly budget and 80% to add up their food shopping
The "What you say counts" campaign will be launched by the minister at a supermarket in Cardiff Bay on Monday.
It aims to counter the view that it is normal or acceptable to be bad at maths, emphasising its role as a vital part of everyday life.
Parents and children will be able to test out their maths skills in calculating the cost of a shopping basket of groceries watched by TV personality Arfon Haines Davies, who is fronting the campaign.