Basically, you will need to override the "Enhanced" editor that TC uses. What we did was copy the code from the CommunityServer.Controls.TinyMCE object in the 5.5 SDK. The issue is occurring in the "IsSupported" method where the hard coded browser names are. You will need to change the line that reads...
if (browser.IsBrowser("Gecko"))
to
if (browser.IsBrowser("Gecko") || browser.IsBrowser("FireFox"))
That is what worked for us. The line number is 106 in the 5.5 SDK.
Next you will need to add the override to the CommunityServer_Override.config file. Something like this...
<Override xpath="/CommunityServer/Core/editors/editor[@name='Enhanced']" mode="change" name="type" value="MyCustomTCLib.MyTinyMCE, MyCustomTCLib" />
If you need more than this just DM me and we can discuss in more detail.
Please just remember Alex's comment above that TC5.5 has not been tested with 4.0 and is not officially supported when using 4.0. So if you plan to request Telligent support you should revert back to 3.5.
Good Luck!