Hi Anders,
Unfortunately even though the RadCaptcha is part of the ASP.NET AJAX Rad controls, the captcha control itself cannot be used via Ajax. I've already confirmed previously on the Telerik forum that client-side validation is not possible. You can take a look at my proposed technique and Telerik support's response as why it does not work.
The code for my RadCaptcha is simple and I really just had to add it to the existing Telligent comment form validation group:
<telerik:RadCaptcha ID="Captcha" runat="server"
CaptchaTextBoxLabel="Enter the characters below into the field."
CaptchaTextBoxLabelCssClass="lblInstructions"
CaptchaTextBoxCssClass="tb"
CssClass="pnlCaptcha"
EnableRefreshImage="true"
Display="Dynamic"
TabIndex="4"
ValidationGroup="CreateCommentForm">
<CaptchaImage
ImageCssClass="imgCaptcha"
TextColor="#000000"
UseRandomFont="true"
BackgroundColor="#ffffff"
BackgroundNoise="Low"
LineNoise="None"
FontWarp="Low"
TextLength="6"
TextChars="LettersAndNumbers" />
</telerik:RadCaptcha>
As you can see, the validation group is "CreateCommentForm"
Good luck!
Mark