You can not delete a Team Project from Source Control Explorer. When you try to delete it from Source Control Explorer in Visual Studio, it says to use team project deletion tool.

You can remove a team project from Team Foundation Server when the project is no longer required by using TFSDeleteProject.exe.

Open Visual Studio 2010 Command prompt and type;

"http://chan08:8080/tfs/defaultcollection" is my tfs 2010 team project collection url.
"demo2" is the team project will be deleted.
"/q" is for brave administrators. :)

tfsdeleteproject /q /collection:http://chan08:8080/tfs/defaultcollection demo2

Deleting from Build ...
Done
Deleting from Version Control ...
Done
Deleting from Work Item Tracking ...
Done
Deleting from TestManagement ...
Done
Deleting from LabManagement ...
  Not found
Deleting Report Server files ...
Done
Detaching SharePoint site ...
Done
Deleting SharePoint site ...
Done
Deleting from Team Foundation Core ...
Done