Keeping an eye on the competition is something a lot of SEOs forget to think about. During the last CEO Swap, Wil Reynolds took a walk in Rand Fishkin’s shoes as CEO of Moz. His biggest takeaway about their new Moz Analytics product was that it is a limited in the ability of monitoring competitors.
If you’re making some headway either with rankings or social content, you want to be able to know when your rival is coming up behind you, trying to beat you to the top.
The best way to do this would be to monitor their social activity and blog wouldn’t it? But who has time to check that stuff all day. If you weren’t aware of IFTTT (If This Then That) before this, get over to their site and make an account and start playing around. Then come back here and learn this great recipe.
Monitor A Competitor’s Blog
Wanna skip to the social section? Click this link.
Your competitor’s blog is probably the #1 source of content you’ll want to keep an eye on. It will most likely give you explicit insight into what keywords they are targeting. For this recipe we’ll be using the RSS, Gmail and Google Drive channels so make sure you have them activated in your IFTTT account.
- Go to your competitors blog and take a quick look for an RSS icon or link. If you can’t locate one, no worries! Copy the URL of the blog and head to Feedburner. If by the time you read this Feeburner is no longer in existence you can try Feedity. Another option is to right click the blog’s page choose “view page source” and then do a find for the term “feed” and see if you can track down a URL. Plug the URL into Feedburner and it will detect the feed URL. Most likely it will be something like http://competingsite.com/feed.
- Go to IFTTT and create a new recipe. The “This” trigger should be an RSS feed (the channel with the RSS icon).
- Next you can choose if you want the trigger to grab every post or only posts with certain keywords. This can be useful if you know you only want to know when they write about your targeted keywords.
- Paste the RSS URL you tracked down into the field.
- Next we will link the feed to an email account. I’m going to use the Gmail channel but you can do this with the Email channel as well. You can customize the body of the email to pretty much anything you want with the different ingredients that IFTTT picks up on. The subject is what you’ll want to be smart about because we will use it later to assign Gmail labels to the incoming emails. You can use something generic like “New Competitor Post” but if you want to categorize incoming posts by company name or niche, then you should label them appropriately:
- Add a description to your new recipe and then head to your Gmail account to finish customizing.
- In your Gmail account, go to Settings and click the Filters tab. Then click “Create a new filter”. The goal here is to assign a label automatically to that incoming email so we can refer to it in the next recipe.
- In the next window add your unique subject line you created in the IFTTT recipe. This tells Gmail to look at all emails with this subject line. I recommend adding quotation marks around the subject so Gmail only searches for that exact phrase.
- In the next window choose to skip the inbox if you want, then check the box called “Apply the label”. Choose your label from the drop-down or choose the “New Label” option to create the new one from this window. When you’re done click “Create Filter” at the bottom.
Now when there is a post from the RSS feed you used, it will enter your Gmail and have a label applied to it automatically. You can manage your labels anytime and they will be stored in the left hand menu of you Gmail account.
Now you have a notification system set up to tell you when a competitor posts something new – which is great! But as all SEOs know, excel sheets are king so we’re going to now setup IFTTT to automatically add those incoming posts to a Google Drive spreadsheet as well.
- Go back to IFTTT and create a new recipe with Gmail as your first trigger.
- From the different scrape types, choose “New email labelled”.
- Enter the same label that you just created in your Gmail account. You may have to create several of these recipes to manage all your labels. If you’re label is “nested” within another you may need to use a backslash (“/”) to separate the parent and child labels.
- Next choose Google Drive and the option “Add row to spreadsheet”. You can try some of the others but this option will keep your posts the most organized.
- Enter a name for your spreadsheet. You’ll notice IFTTT says it will create the spreadsheet if it doesn’t already exist. This recipe seems to work better if the spreadsheet doesn’t already exist.
- Create a folder path if, you’d like, for the location of your spreadsheet. Each item separated by a backslash will be a separate folder. Make sure you don’t begin with a backslash.
- The last section formats the row. Separate cells with 3 pipes i.e. – ||| (shift + forward slash). Play around with the formatting if you like. I definitely recommend not adding the ingredient “BodyPlain” as I did in the screenshot below. This will add the entire body of the email (aka the whole blog post) to an spreadsheet cell. A link to the blog post will suffice in this case.
- Create the recipe
If you used a competitor’s RSS you’ll have to wait until they post again to test the whole process so I would hook up your own blog rss and test it out. The new blog post should show up in your labelled folder in Gmail and should also be added to the new spreadsheet. Magic!
Monitor A Competitor’s Social Channels
Almost just as important as your competition’s blog is their social media presence. Social posts are most likely more frequent and more targeted, giving you further insight into their overall online strategy. Using the same process as above, you can monitor the following social channels of any user through RSS.
First go to http://findmyfacebookid.com/ and plug in the profile or page URL that you want to track. Then copy the Facebook ID it gives you and paste it into this URL:
The RSS format for all pins by a specific user is:
The RSS format for the pins of a specific board is:
The board ID is just the sub directory of the user. So if you grab everything in the board’s URL after the username you’ll be good. For example if I wanted the RSS for Moz’s “Roger” board which is located at http://www.pinterest.com/mozhq/roger/ the username is “mozhq” and the board ID is “roger”.
To get an RSS feed for a Google+ profile, just visit gplusrss.com. If the user has a vanity URL (which are becoming more and more popular) I’m not sure what to do yet and I will update this post when I do.
The IFTTT channel for Instagram lets you monitor the posts of any user so make it your first trigger instead of the RSS channel.
The IFTTT channel works for any Tumblr blog but you have to be following the blog. If you despise your competitor so much that giving them that one extra follower chews you up inside, then just append /rss to the end of their URL and voila! i.e. – mycompetitor.tumblr.com/rss
The Twitter channel of IFTTT only allows you to use actions from your own account and unfortunately the new version of Twitter’s API does not use RSS for its feed output (I believe it uses JSON). I couldn’t find a quick way to access the feed so if anyone knows how I’m sure we’d all appreciate your insight.
IFTTT is a powerful tool and a very beautiful look into the world of scraping data. My recommendation to you is take just a few hours one day and set up a few recipes for monitoring purposes and then leave them alone. There are so many other factors to keep in mind when monitoring your competition so don’t put all your eggs in one basket.
If any of you have other recipes that would work for SEOs trying to automate their campaigns, please share them in the comments so we can all benefit from your genius!