How do I get articles included in current awareness aggregators like Vable?

I want you to become the trusted advisor for IT and Business Development teams, as well as your other end-users. It is important for you to appreciate the technology that news monitoring platforms and content aggregators like Vable rely upon to pick up and share content. 

What if you visit your organisation’s website - or indeed any other website - and see that there are no RSS feeds, no obvious news pages, nor options to sign up for email updates? What are your options? This is why you need to understand the what, why, and how behind the scenes of a content aggregation service.

How does Vable technology work?

I asked our experts and they were happy to explain how they extract and enhance your vital content.

RSS feeds are the best

Current awareness platforms are designed to handle incoming content in many formats. Our system visits web pages and RSS feeds which have been added as sources to the platform and looks for content to pick up based on rules we have set up. From a technological point of view, RSS feeds are the most convenient way to add content and there are handy Chrome extensions to help you find them.

• With RSS feeds, these rules are automatic, because RSS feeds follow standard formatting rules 
• With web pages, a human being must review the page and tell the system what it should be picking up 

Given that email remains a consistent way of alerting people to new content on websites, Vable has designed an efficient way of extracting this information - from attachments, weblinks or just a single item of news, this can be added directly to your current awareness platform. It used to be a challenge to extract information from emails, but with solutions such as Vable Inbox, this is not a problem.

Efficient database searching requires the consistent organisation of information. This is why we try to expand on what is available; we check for available metadata, categories, summaries, and full text (where available) and populate relevant fields. We also analyse incoming English language content with natural language processing and add keywords which describe it. Everything the information professional needs for accurate searching.

The most important thing to remember is that Vable identifies and understands content, by default, by its link; each unique article requires a unique link

What makes your website trackable? 

Be aware of your website’s format

• Most importantly, a site must have an updating list of content. This can be a news, topic, press release, or events page with links that can be picked up directly from the page. Remember each article should have a unique weblink.
• Make this page easy to find, and ideally ensure it loads without any additional/complex scripts. These can slow down the loading of the content onto the page and can cause issues for crawlers.
• Ensure your articles are in date order, with the latest information at the top of the list. If there is more than one list of items, separate them into columns, or in some other significant and clear manner. 
• For the benefit of your users, always add a clear date of publication on articles. This also helps us when bringing in new content. 
• Always provide unique links to new articles. If you need to make minor corrections to the text, you don’t need a new link. But if it’s substantially different and you want to draw attention to it, then provide a new updated link for us to include.

Keep your content options open

• Have important content in more than one place on your website. For example, if there is an issue tracking the news from a search results filter because it takes too long to load, we could find the latest news on the homepage instead.
• The more options on the site the better! Include the content on a dedicated news page, but also provide an RSS feed. Ensure total flexibility by offering an email subscription. We recommend an RSS feed - or preferably multiple topic or events feeds - to make your content visible to crawlers.

Ensure that you permit crawlers

Don’t block crawlers. If you want your content to be visible to crawlers and included in news aggregation platforms, do not have anything on your website which blocks them! This includes robot exclusion rules, security software like Cloudflare, “Are you a robot?” prompts requiring tick-boxes to be selected or questions to be answered.

Additional tips for the technology professionals in your firm

JavaScript can cause delays

JavaScript is often used on law firm websites. We find that in some cases this can cause issues with crawling. It can create a delay in loading content, cause a timeout or prevent our system from seeing the content in some circumstances. Following JavaScript best practices should ensure even complex content is as accessible as possible.

Update your RSS feeds 

If using an RSS feed, don’t neglect it.  Making sure they are updated in tandem with the web page, as we sometimes find out of date feeds, even if the site is current. 

Make use of your search functionality

Adding search functionality to your website can be an excellent way of making the most of your content. We recommend you set up search engines to return results with unique links provided for a given result, that is to say, parameters are added to the browser URL for the keywords and filters used. This allows users to treat a search on your page as a unique source in our platform and we can alert them when new results match their search. Better yet, allow users to create custom RSS feeds from the results.

Once you have all your sources in the system, Vable monitors them to ensure continuous content. Our system assigns them a ‘health’ status based on various parameters which is updated each time our web crawlers visit it. This status corresponds to the state of the source and its content during the most recent visit. 

Premium content aggregator sites like Vable are a great current awareness tool for library and information professionals. We want to enter 2021 prepared for anything and with a brand new business development strategy. Is it possible that content aggregation could play a part in the ongoing success of your firm?

When your firm’s experts spend time creating valuable original content, it must be read and shared to make the desired impact. Your content deserves the best tech; and the tech deserves the best content. If you have any questions, we are always on hand to extract, expand and enhance!