Scoring

Overview of the Scoring service

New to the Telligent Evolution Platform is the scoring service. This service is responsible for calculating scores for content items. Out of the box, Telligent Community and Telligent Enterprise both use the scoring service for determining content quality and for handling abusive content.

Scores represent a behavior that you want to encourage (creating quality content) or discourage (creating abusive content). Metrics give value an an event (liking, for example) on a score (an average of all the metrics) and the scoring service is the way that you calculate the result of the behavior you are encouraging/discouraging.

At its core, a score is a plugin that knows how to calculate an aggregate value for a given piece of content. A score is a collection of metrics and a calculation method which, together, are used to create a value. By default, score values are simply a weighted average of metric values, but the calculation method can be changed as necessary. An example of a score used in the platform would be the Content Quality score.

Similar to a score, a metric is a plugin that knows how to calculate an atomic value for a given piece of content. A metric contains a calculation method and attaches to events which govern when a metric is recalculated. An example of a metric used in the platform would be the Like metric.

Recalculation

When a score is recalculated, it's important to note that the underlying metrics are *not* recalculated. While the score's calculation method is run, *existing* metric values are used in the calculation.

Scores are recalculated by the platform in the following scenarios:

Whenever a metric value changes
Whenever a metric is added to or removed from a score
Whenever metric weights are changed
When the half-life decay interval expires (default = 30 days)

When a metric is recalculated, the metric's calculation method is run. Since a metric is the atomic unit in the scoring service, the platform never triggers a metric recalculation. Only events that the metric is subscribed to can trigger a recalculation.

Recalculation is a two-step process. In the first step, a score or metric is simply "flagged" for recalculation. The actual recalculation occurs in the second step, when the score or metric recalculation job runs. Until the recalculation occurs, the existing score or metric value will be used by the platform.

Jobs

Score Recalculation job

By default, the Score Recalculation job is run every 15 minutes. Its purpose is to recalculate all scores that have been flagged as requiring recalculation.

Metric Recalculation job

By default, the Metric Recalculation job is run every 15 minutes. Its purpose is to recalculate all metrics that have been flagged as requiring recalculation.

Score Decay job

By default, the Score Decay job is run once a day. Its purpose is to trigger a score recalculation for all scores whose decay interval has "expired." So while the job runs daily, a particular score would only be recalculated for decay on its decay interval (default = 30 days).

Decay

There are some scores that are meant to decrease over time. For such scores, decay should be enabled. The current method for decay in the scoring service is half-life based. That is, if a score enables decay, its value will be cut in half at each decay interval. By default, the decay interval for a score is 30 days. So if a score value is 0.8 at a given point in time, that value will be 0.4 when decayed 30 days later. This method of decay ensures that more recent scores are higher while older scores decrease gradually over time.

Decay can be enabled on a score-by-score basis. Furthermore, the decay interval can be configured at the score level and at the application level. That is, while a score may decay every 30 days by default, it is possible to configure that same score to decay faster or slower in a particular blog, forum, gallery, etc.

In Telligent Community and Telligent Enterprise, the Content Quality score has decay enabled by default. Essentially this means that more recent content will have a higher content quality score that older content. Other scores have decay disabled by default.

Weights

Since a score can be made up of multiple metrics, it's important to be able to value one metric more than another. This is the role of weights. By default, each metric in a score is weighted equally. Metric weights can be changed in the control panel or the default weights can be changed in the metric plugin itself

How tos

When considering creating a new score, ask yourself "What behavior am I trying to change in the community?". If the answer is quantitative, like "I want community members to create more forum threads", then really think about that. A single user could create 100 obnoxious forum threads and therefore your community is not very useful.

How to configure scores

See Score and Metric configuration for more information.

How to create a new metric

A metric is a plugin, which means it needs to implement the IPlugin interface. In addition, it needs to implement the IMetric interface.

public interface IPlugin
{
    string Name { get; }
    string Description { get; }
    void Initialize();
}

public interface IMetric : IPlugin
{
    Guid Id { get; }
    string MetricName { get; }
    string MetricDescription { get; }
    double Calculate(Guid contentId, Guid contentTypeId);
    void SetController(IMetricController controller);
}

The key areas to focus on are the Initialize() and Calculate(Guid, Guid) methods. Typically, the Initialize() method is where you'll subscribe to events that will trigger a recalculation. For example, the CommentsMetric subscribes to both the AfterCreate and AfterDelete events. When either of those events are fired, the metric queues a calculation for the content that was commented on. The Calculate(Guid, Guid) method is where you'll actually determine the value for the metric. This can be as simple as a database lookup or as complex as a webservice call to an external service. Again, in the CommentsMetric, the Calculate(Guid, Guid) method simply calls into the CommentsService, gets the number of comments for a particular piece of content, and then normalizes the value.

How do I add a custom metric to an existing score?

Compile the metric into an assembly.
Add the assembly to website's /bin directory.
Enable the metric plugin in Control Panel > Site Administration > Site Configuration > Manage Plugins.
Go to Control Panel > Site Administration > Site Configuration > Score Configuration applet and select the target score (on the left).
Scroll to the bottom of the list of metrics (on the right) and click Add/Remove.
Move the metric from the list of available metrics to the list of active metrics and click Save.
Adjust the metric weight(s) as necessary.

See Core Metrics for a list of predefined metrics that can be used in custom scores.

How to create a new score

A score is a plugin, which means it needs to implement the IPlugin interface. In addition, it needs to implement the IScore interface.

public interface IPlugin
{
    string Name { get; }
    string Description { get; }
    void Initialize();
}

public interface IScore : IPlugin
{
    Guid Id { get; }
    string ScoreName { get; }
    string ScoreDescription { get; }
    bool IsDecayEnabledByDefault { get; }
    bool AreDecayOverridesEnabledByDefault { get; }
    int DefaultHalfLife { get; }
    IEnumerable<Guid> DefaultMetricIds { get; }
    void SetController(IScoreController controller);
}

The key areas to focus on are the DefaultMetricIds property and the decay related properties. The DefaultMetricIds enumerates the list of metrics that are included in the score calculation by default. The decay related properties should be self explanatory. Just remember that they are all defaults and can be overridden by community managers in the Control Panel. If there are specific scenarios when you want your score recalculated, you can subscribe to specific events in the Initialize() method and queue the calculation. By default, a score is recalculated whenever a contained metric value changes.

Since the platform provides a default implementation for the calculation of a weighted average score, you do not need to implement a calculate method in your score plugin. If a weighted average is not the right method for calculating your score, and if you want to customize how your score is calculated, you can alternatively implement ICustomCalculatedScore.

public interface ICustomCalculatedScore : IScore
{
    double Calculate(Guid contentId, Guid contentTypeId, IEnumerable<IWeightedMetric> weightedMetrics);
}

Implementing this method means that you are responsible for gathering metric values, applying weights, and determining the final score value.

Finally, if you only want your score to be calculated on a particular content type, you can alternatively implement IContentTypeLimitedScore.

public interface IContentTypeLimitedScore : IScore
{
    IEnumerable<Guid> SupportedContentTypes { get; }
}

If you implement this interface and a score calculation is requested for an unsupported content type, then an exception is thrown.

How do I add a custom score?

Compile the score into an assembly.
Add the assembly to web site's /bin directory.
Enable the score plugin in Control Panel > Site Administration > Site Configuration > Manage Plugins.
To optionally configure the score, go to Control Panel > Site Administration > Site Configuration > Score Configuration applet and select the target score (on the left).

How do I use custom scores?

Scores are used as a sort parameter when getting a list of content, people, or applications.

Score_ID is exposed on these endpoints:
List forum thread
List forum reply
List blog post
List media (post)
List comment
List Group User
List Blog
List Forum
List Gallery
List Group
List Media
List Wiki

Pass the Score_ID of your desired score in the form of its GUID to sort the lists above.

Developer Space