At the Fed Cup draw in Constanta on Friday before a tie between Romania and Britain, Nastase was heard speaking in Romanian to one of his team members about Williams' baby.
"Let's see what colour it has. Chocolate with milk?" he was quoted by Romanian and some British media as saying.
Williams, who confirmed her pregnancy through a spokeswoman on Wednesday, is engaged to Reddit co-founder Alexis Ohanian, who is white.
Nastase, who was later kicked out of the tie by the ITF for "unsportsmanlike conduct" on court, told Romanian television news channel, Digi24, that his remark is not racist and that he has a very good relationship with Williams.
"This was a joke," he said. "If someone didn't understand it, they lack humour... they have their own humour and we have our humour."
The ITF said in an emailed statement it does not tolerate discriminatory and offensive language.
"We are aware of alleged comments made by Romanian Captain Ilie Nastase and have begun an immediate investigation so that we have the full facts of the situation before taking further and appropriate action."
When play began on Saturday, Nastase was soon in trouble on court and was sent off by the umpire in the second rubber between Britain's Johanna Konta and Romania's Sorana Cirstea for "unsportsmanlike conduct". His exclusion from the tie followed.
Nastase was heard making derogatory remarks that appeared to be aimed at the umpire and at Konta. The match was temporarily suspended with Konta in tears, before restarting shortly afterwards. Konta won the match 6-2, 6-3 to level the tie at 1-1.
"Mr Nastase was also removed from the grounds due to his serious misconduct," the ITF said in a second statement.
"His accreditation was removed and he will play no further part in the tie. The ITF has launched an investigation into this matter as well as previous comments made by Mr Nastase during the week."
Image: Ilie Nastase
Photograph: Fadi Al-Assaad/Reuters