how do I get a rgb pixel image matrix for a simle motion tracker program?

First I am new to blender python scripting so forgive me if some questiones are stupid. I am trying to make a simple motion tracker program that is able to read out the rgb values of the pixels in an image or movie. It should at least be able to search the values of a predefined area (matrix or list) compatre them and output the frame number and 2D-positions. So they may be attached to an empty or hook or something like that. Is there a way to get the needed image or movie data?