Prime Cabinet Secretary Musalia Mudavadi has distanced the government from Trade CS Moses Kuria's attack on media.
He said if someone feels aggrieved, they should criticise and object but using vile language is not an option.
Mudavadi's sentiments come after public uproar following Kuria's derogatory description of Nation Media Group.
In an interview with NTV on Tuesday night, Mudavadi said Kenya must always have vibrant and free media as per the Constitution.
"The government is committed to a free, vibrant and objective media and we are not running away from that. There is a thin line between a public official and someone in their private capacity," he said.
"My advice to my colleagues in public space is that, be careful in the statements that you make because they may be personal statements but because of your status in society, they could give the impression that it is public policy, which is not."
Kuria launched an attack on the Nation Media Group over the weekend following an expose by NTV that touched on the conduct of Cabinet secretaries.
The TV station alleged in its report that Cabinet secretaries engineered a drop in the prices of edible oils resulting in the loss of taxpayer's money to a tune of about Sh5.6 billion.
Kuria threatened the media organisation using unprintable words and issued a threat to state agencies against running adverts with the media organisation.
Media stakeholders such as the Kenya Editors Guild, Kenya Union of Journalists condemned CS Kuria saying his actions and utterances should promote a positive image of Kenya as a nation.
They also called on President William Ruto to act following the remarks, adding that leaders are the mirrors of society.
The Ethics and Anti-Corruption Commission rubbished Kuria’s threats that he will sack any state official who would dare place government adverts with the media organisation.
In a statement, EACC Chief Executive Officer Twalib Mbarak said no public institution or public official should be victimised for engaging in any lawful dealings with any organisation, including in the award of tenders.
On Monday, Deputy President Rigathi Gachagua also distanced government from Kuria's remarks saying the government will allow the media to work freely although they won't hesitate to call it out should it go overboard.