Monday, 13 October 2008

Capturing an image/ thumbnail of a webpage in C#

For a while I've been figuring out how to programmatically get an image of a web page using C# and .Net. This could have a number of uses such as displaying a thumbnail of a web page. I found a number of methods by Googling but on the whole they seemed a bit lengthy. Eventually, I combined bits of a number of methods and then simplified things by trying out an alternative approach myself. One key thing I wanted to do was to create a mini picture of a website for display in a new desktop app I'm developing.

In the end I wrote a simple class with a single static method called
GrabImageOfWebPage. GrabImageOfWebPage takes a .Net WebBrowser control instance as an argument together with the required size for the captured image. The web page loaded in the WebBrowser control is captured (the entire client area of the control is captured) and shrunk/ enlarged into a bitmap of the required size. Here's the code:


using System;
using System.Collections.Generic;
using System.Text;
using System.Windows.Forms;
using Microsoft;
using mshtml;
using System.Runtime.InteropServices;
using System.Runtime.InteropServices.ComTypes;
using System.Drawing;

namespace BrowserComponents
{
///
/// Class providing a static method to return a bitmap of a web page rendered in
/// a .Net webbrowser control.
///

public class CBrowserImageGrabber
{

[ComVisible(true), ComImport()]
[GuidAttribute("0000010d-0000-0000-C000-000000000046")]
[InterfaceTypeAttribute(ComInterfaceType.InterfaceIsIUnknown)]

private interface IViewObject
{
[return: MarshalAs(UnmanagedType.I4)]
[PreserveSig]
int Draw(
//tagDVASPECT
[MarshalAs(UnmanagedType.U4)] UInt32 dwDrawAspect,
int lindex,
IntPtr pvAspect,
[In] IntPtr ptd,
//[MarshalAs(UnmanagedType.Struct)] ref DVTARGETDEVICE ptd,
IntPtr hdcTargetDev,
IntPtr hdcDraw,
[MarshalAs(UnmanagedType.Struct)] ref tagRECT lprcBounds,
[MarshalAs(UnmanagedType.Struct)] ref tagRECT lprcWBounds,
IntPtr pfnContinue,
[MarshalAs(UnmanagedType.U4)] UInt32 dwContinue);
}

public static Image GrabImageOfWebPage

(WebBrowser Browser, Size ImageSize)
{
// Get the view object of the browser
//
IViewObject VObject = Browser.Document.DomDocument as IViewObject;


if (VObject != null)
{
// Construct a bitmap as big as the required image.
//
Bitmap bmp = new Bitmap(ImageSize.Width, ImageSize.Height);


// The size of the portion of the web page to be captured.
//
mshtml.tagRECT SourceRect = new tagRECT();
SourceRect.left = 0;
SourceRect.top = 0;
SourceRect.right = Browser.Right;
SourceRect.bottom = Browser.Bottom;



// The size to render the target image. This can be used
// to shrink the image to a thumbnail.
//
mshtml.tagRECT TargetRect = new tagRECT();
TargetRect.left = 0;
TargetRect.top = 0;
TargetRect.right = ImageSize.Width;
TargetRect.bottom = ImageSize.Height;



// Draw the web page into the bitmap.
//
using (Graphics gr = Graphics.FromImage(bmp))
{
IntPtr hdc = gr.GetHdc();
int hr =
VObject.Draw((int)DVASPECT.DVASPECT_CONTENT,
(int)-1, IntPtr.Zero, IntPtr.Zero,
IntPtr.Zero, hdc, ref TargetRect, ref SourceRect,
IntPtr.Zero, (uint)0);
gr.ReleaseHdc();
}



// Return the bitmap.
//
return bmp;
}
else
{
return null;
}
}
}
}


}
To gain visibility of the types in this example, you have to include the following uses:
using mshtml;
using System.Runtime.InteropServices;
using System.Runtime.InteropServices.ComTypes;
using System.Drawing;
and you also need to add a .Net reference to your project for Microsoft.mshtml.

Using the method is then pretty easy. The example code below create a webbrowser control and loads a webpage. When the webpage is fully loaded, it grabs an image 10% the size of the original page and displays it in a picture box.

WebBrowser mWebBrowser;

public Form1()
{
InitializeComponent();

mWebBrowser = new WebBrowser();
mWebBrowser.Width = 1024;
mWebBrowser.Height = 768;
mWebBrowser.ScrollBarsEnabled = false;

mWebBrowser.DocumentCompleted +=
new WebBrowserDocumentCompletedEventHandler
(mWebBrowser_DocumentCompleted);
mWebBrowser.Navigate
(@"http://www.software-product-development.blogspot.com");
}


void mWebBrowser_DocumentCompleted(object sender,
WebBrowserDocumentCompletedEventArgs e)
{
if (mWebBrowser.ReadyState == WebBrowserReadyState.Complete)
{
Image Img =
BrowserComponents.CBrowserImageGrabber.
GrabImageOfWebPage(mWebBrowser, new Size(102, 77));

if (Img != null)
{
pictureBox1.Image = Img;
}
}
}

Monday, 6 October 2008

Losing a PageRank Value

In the mini-update toolbar export on Sept 26th this blog lost its PR, i.e the value went to N/A. Previously the PR had been 3. Why this has happened I don't know. I haven't linked to any silly sites or reduced the frequency of posting.

Today I also noticed that a few inner pages on the blog have PR. I haven't noticed this before. It's strange that the older posts have PR but the blog home page is back to PR N/A. Google Analytics isn't showing any change in the level of traffic to the blog.

Friday, 12 September 2008

How to keep on the good side of Google

When people are trying to improve their ranking in search engine results page, a lot of people will use any advice they can find in a struggle to improve rankings. People need to be wary which advice they follow though. Some advice will cause you to be penalised by Google.

If a page on your website is penalised, it will not perform as well as it might in search results. Here are some tips to help avoid penalties. In the case of a new website, it may never get to perform well in the first place.

Remember that outside Google, no-one really knows what counts as good or bad in terms of SEO. There is some general advice from Google about having good, unique content and quality backlinks. Other than that, people are justing using their experience and guesswork to find out what works and doesn't. A lot of SEO information online is copied and spread as online myths.

With this proviso in mind, here's a fairly non-controversial list of things to avoid.

  • Avoid Exchanging Links
    Excessive link exchanging should be avoided as Google may see this as an attempt to artifically improve rank. A few link exchanges will be OK but avoid large numbers. Link farms - where a large group of sites hyperlink to all other sites in the group should always be avoided.
  • Do not Sell links
    Selling links is a no-no - unless the hrefs use the nofollow attribute. If your site sells dofollow links and Google becomes aware it may well be penalised. Google's WebMasters site allows people to report paid links. Rumour has it that Google may use this information to adjust its algorithms to improve detection of paid links.
  • Do not buy links
    Recently, Google has threatened to penalise sites they discover have purchases dofollow links from another site. Logically thinking though, this does not seem possible - or at least it would be extremely unfair! If this were to be the case it would be easy to penalise a competitor by purchasing links to their site and then reporting them to Google.
  • Avoid duplicate content
    If possible, avoid duplicate content. For example, don't make the same post to two different blogs. Google will ignore copies of content. A few copies will make it into Google's index but lots of copies will be ignored. Even on different pages within a website, try to keep the textual content unique and avoid repeating whole paragraphs of text.
  • Don't stuff keywords
    If you want to perform well for a certain keyword, stuffing your webpages full of the keyword will not help. Write you copy in a natural way so that it reads well. If you are writing a web page about Google penalties (for example), the keyword "Google Penalty" will naturally appear a number of times - you don't need to repeat the phrase scores or hundred of times.
  • Don't include hidden text
    Make the contents of the webpage visible to the user. For example, don't include extra content such as white text on a white background that the user cannot see.

Sunday, 7 September 2008

Sitelinks update

Today, Google Webmasters shows an updates in my sitelinks but these don't seem to have made it to the SERPs yet. Three new links have been added to my sales, support and SliQ Submitter pages. When these links make it to the search results, the sitelinks will look much better than at present.

Sunday, 31 August 2008

Further thoughts on Google Sitelinks

Thinking about it a bit more, my feeling is that Google needed knowledge of where people went after landing on the SliQTools homepage in order to choose the sitelinks. The only way Google could have this knowledge is by using Google Analytics data. I wonder if sitelinks have ever been given to a site that did not use Analytics?

One of the search terms for which sitelinks are displayed is used by people revisiting the site to see if there is a new release. After landing on the homepage, they go to the Release History Page.

I did a rejig of the homepage a couple of weeks ago and shuffled around some of the thumbnails to make people go to a better page after the homepage, i.e. the Invoice Software page instead of the Invoices & Payments page. I think the Invoices & Payments sitelink was generated before this reshuffle. Perhaps after 90 days the sitelinks will be regenerated inline with the new site linking stucture.

SliQTools now has SiteLinks

Today I noticed that for a couple of search terms, the SliQTools website now has sitelinks on Google. The links aren't really to the pages I would have wished to highlight to users.

If I could generate the sitelinks manually - which isn't possible with Google sitelinks - I'd have chosen the invoice software page, download trial page, invoice templates page and sales pages. Instead, Google has chosen the Invoices & Payments page, Invoice Templates page and Release History pages.

It's interesting how Google has chosen the names for the links too.

The names aren't the first words in the page titles, i.e. in the HTML title tags. Also, in the case of the Invoice Templates link, the link text isn't the anchor text used for links to the pages within the site either. Interestingly, the Invoices & Payments link has only one text link pointing to it within the SliQTools website - mostly it's linked from a thumbnail. I thnk the sitelinks persist for 3 months, i.e. the links won't change for 90 days.

Monday, 25 August 2008

Different sites, different visitor profiles

I thought I'd make a record of the share of visitors two of my sites get from different search engines. The profiles are very different and not really what I'd expected. the first sets gets over 90% of its traffic from direct addresses/ bookmarks but here are its search engine stats:-

  1. Windows Live - 53%
  2. Google - 32%
  3. Yahoo - 10%
  4. AltaVista - 2%
  5. Alexa,Google Images,Tiscali,Dogpile,AllTheWeb - 3%

The second site gets less than 5% of visitors from direct traffic. Its search engine stats are:-

  1. Google - 77%
  2. Yahoo - 10%
  3. Unknown - 8%
  4. AOL - 2%
  5. MSN, Tiscali, Ask Jeeves, Alexa, AltaVista - 3%

The search engine stats are very different between the sites. The second site is my oldest. For both sites Google is very important but I'm surprised to see Windows Live heading up the list for the first site. The first site is quite new - less than 2 months old. In another couple of months I'll see how the search engine stats are looking then.

Saturday, 23 August 2008

Google PageRank - Second Update

I'd been confused by the visible Pagerank of this blog as shown on the Google toolbar. After the last export of rank values at the end of July this blog had still not been assigned a rank value by Google and was still showing PR N/A. However today - Aug 23rd - the rank is showing as PR 3! This is great news and much more in line with what I was expecting. I'd been doing a mixture of different types of link-building and had also been doing directory submissions using SliQ Submitter, my directory submission software. This now looks as though it has paid off - at least in terms of PR - and ties up with the increase in the number of visitors to the blog over the past 4 weeks.

Thursday, 21 August 2008

Tracking performance in SERPS for keywords

A few day ago I was speaking to a friend who was testing out SliQ Submitter for me and providing some useful, constructive feedback. He was wondering what tools I used to check the ranking of my site on Google for different keywords. I don't use anything other than Google searches by hand but I thought I'd look to see if there was anything on the market.

I soon found a tool called WebPosition Gold or WPG for short. This tool allows you to automatically track the performance of sites on Google and show nice graphs of the historical trend. Trouble is that with a bit more reading online, it turns out that Google mentions that use of WPG is against Google's terms of service. the terms specifically name WPG as well as saying that similar tools are also not to be used.

To use WPG you have to a Google API key for their SOAP search API. Google no longer issue such keys and there are stories online saying that WPG users are now finding that WPG gets blocked. The future doesn't look good for WPG.

When I think about, it seems fairly clear that tracking a few keywords is interesting but not necessarily informative anyway. What really matters is traffic to your site and conversion of visitors to sales. You could be performing really well for certain keywords - and improving in performance too - but really you need to measure traffic. Google itself provides a better measurement tool - Google Analytics. If you have the time, Google Analytics can provide valuable data about the performance of your website - traffic figures, how people got to your site (not just SE performance) and where they went went they landed on your site.

Monday, 18 August 2008

SliQ Submitter - New Release

SliQTools have released an updated version of their free directory submission tool SliQ Submitter.

After feedback from beta testers, a number of enhancements have been made:-
  1. SliQ Submitter now attempts to learn directory categories selected by the user on the directory page. This means that the user is no longer restricted to a small, fixed set of categories to be filled in automatically - the more categories you use when submitting to directories, the more categories SliQ will learn.
  2. If SliQ Submitter can detect that a website has been successfully submitted, SliQ will load the next directory automatically.
  3. A number of GUI enhancements have also been made.

The end results of these improvements is that SliQ Submitter is now easier to use and makes the process of web directory submission faster.

Read the original release post for SliQ Submitter.