Some PeerIndex users have notice recent changes in their ranks. These changes have occurred because of some recent updates we’ve made to the one of the core algorithms we use to determine your PeerIndex.
In doing these changes, we have failed to explain clearly the possible effects on your ranks which left you in the dark trying to understand what is going on. The simple fact is I screwed up in explaining the likely effects of the changes would be. My apologies.
The rest of the post will explain more about this one algorithm: specifically the topic resonance algorithm that helps us identify whether someone is an authority in a given topic. We use other algorithms, which are used to identify engagement levels within your audience, the quality of your network, etc, but I won’t go into them much in this discussion. I’ll also overview the recent changes, along with other updates that will be rolled out over the next few weeks.
How our resonance ranking algorithm works
The ranking works by building a resonance network for each of the nine benchmark topics along with a wider range of 1500+ topics.
The resonance network is effectively a PageRank-like algorithm that works on a graph. This graph is created by looking at the diffusion of actions both within your first-order social network and the global network. We look at the diffusion of actions across both Twitter and Facebook streams.
This graph is built out of 120 days of actions associated with a topic. The 120 days is a sliding window and goes backwards from the date of calculation. So your rank will rise and fall as your activity rises and falls through the 120 day sliding window. This four-month time period is long enough to ensure that authority grows and is consistent, but short enough to demand that you remain active over this period.
The score produced by the resonance network is then normalised across all the people in the topic to produce the familiar rank. The normalisation is non-linear, so to get a 90 you need to be in the top 0.001% of the community. In a community of 1000 people that means only 1 can receive a 90+. The normalisation means that your rank changes as your activity changes relative to the community.
Your final overall authority is produced as a weighted sum of the benchmark topics and the other topics with the benchmark resonance being weighted more. The final PeerIndex is produced by adding in other outputs, which we call audience, and activity, that are generated by other algorithms. These variables include measures that weight the quality of your network, the people you most engage with, how many people you engage with and so on.
Why your resonance / authority rank can change naturally
You rank can change for the following reasons:
- Your activity not longer resonates with the community
- Your activity falls relative to the rest of the community in the 120 day window (e.g. you went on holiday for 2 weeks or there was sudden surge in activity from the rest of the community that you didn’t participate in)
- The size of the community increases dramatically (i.e. went from 1000 to 10,000 between calculations) as the size of the community increases the greater the chance there is someone who is more resonant than you. In practice, this would not happen very quickly. Communities tend to have relatively stable sizes on a week-to-week basis.
Recent changes to the algorithms
As I stated at the beginning of the post, we made some recent changes to the algorithms that have produced their own effects to change users ranks.
These changes are:
- We refined the benchmark topics from an original 16 to the current nine. We felt the nine new benchmarks better captured activity. During this transition we remove some double counting of topics.
- Added hysteresis to the calculation. The effect of which is to slow the propagation of changes to ranks (whether going up or going down)
- Changed the weighting between the benchmark & other topics. We are in the gradual process of fine-tuning the weighting.
Of the three changes, the changes to the benchmark topics produced the largest effect, mainly from the removal of the double counting that crept in as we transitioned to the new benchmark topics.
Removing the double counting will produce the result of lowering overall authority and consequently PeerIndex. The hysteresis has produce a gradual fall of ranks over the appropriate time period.
We’ve got additional changes lined up over the next few weeks. These changes will focus on:
- The core resonance algorithms
- Improved the use of the data from Quora, blogs & and LinkedIn
- Adjustment of normalisation
We are updating the resonance algorithm to take better into account engagement, your social graph and activity in the topic. We are making these changes as our testing has shown it improves the rankings of those who are engaged but not necessarily very active, the quiet gurus if you want. We’ve also found that it goes a very long way to reducing the noise in the rankings. People will rise in rankings and spambots will fall.
We are adjusting the normalisation to take into account the increased number of signals and data we are using to produce the rankings.
Improving your rank with these changes
The reasonable question to ask is how can I improve my ranking in the face of all these changes? In general you can do the following things:
- Be engaged in the community around the topics you are interested. By this I mean you need have conversations with others interested in the topic. Simply @’ing or RT’ing people in the community is not enough, you need to engage in on-going conversations with a broad range of people in the community.
- Target your activity to the topics you are most keen in and make sure you are sharing content that resonates (i.e. the rest of the topic community find interesting). Don’t spam as the new algorithms will punish.
- Encourage your followers who are also interested in the topic to join PeerIndex. This will increase the chance we will identify accurately that your followers are interested in a topic
- Add in your profiles from other channels.
We will be rolling out personalised suggestions on how to improve your rankings and an indication of the value various actions will have on improving your rank over the next month to two months.
Our aim is to produce the most reasonable model of authority and reputation given the available data sources. This is so that you can benefit from the social capital that you’ve accrued and so that you can better understand with whom you are interacting.
We will continue to roll out refinements for the algorithms. Our lesson learned from the recent changes is that we have to communicate these changes early and clearly so you are not taken by surprise.