Introduction
I was busy trying to write a motion detection algorithm in C#. To determine motion, I compared change values from a previous image by calculating the absolute value of the pixel difference. For doing this, grayscale is easier and probably faster.
Background
If MIT thinks using grayscale for motion detection is a good idea, then I am definitely on the right track with my motion detection algorithm. For a reference, see the Cog project.
I got the code for the image loop from here because I originally attempted to use SetPixel which is painfully slow.
Using the code
If you use this code in a multithreaded environment, be aware that the parameter image will be locked until this method unlocks it. This will raise an exception when another code attempts to lock the bitmap, such as a customer OnPaint. To avoid this, you can always make a copy of the bitmap.
Here is the entire method:
Collapse Copy Codepublic Bitmap processImage(Bitmap image){
Bitmap returnMap = new Bitmap(image.Width, image.Height,
PixelFormat.Format32bppArgb);
BitmapData bitmapData1 = image.LockBits(new Rectangle(0, 0,
image.Width, image.Height),
ImageLockMode.ReadOnly,
PixelFormat.Format32bppArgb);
BitmapData bitmapData2 = returnMap.LockBits(new Rectangle(0, 0,
returnMap.Width, returnMap.Height),
ImageLockMode.ReadOnly,
PixelFormat.Format32bppArgb);
int a = 0;
unsafe {
byte* imagePointer1 = (byte*)bitmapData1.Scan0;
byte* imagePointer2 = (byte*)bitmapData2.Scan0;
for(int i = 0; i < bitmapData1.Height; i++) {
for(int j = 0; j < bitmapData1.Width; j++) {
// write the logic implementation here
a = (imagePointer1[0] + imagePointer1[1] +
imagePointer1[2])/3;
imagePointer2[0] = (byte)a;
imagePointer2[1] = (byte)a;
imagePointer2[2] = (byte)a;
imagePointer2[3] = imagePointer1[3];
//4 bytes per pixel
imagePointer1 += 4;
imagePointer2 += 4;
}//end for j
//4 bytes per pixel
imagePointer1 += bitmapData1.Stride -
(bitmapData1.Width * 4);
imagePointer2 += bitmapData1.Stride -
(bitmapData1.Width * 4);
}//end for i
}//end unsafe
returnMap.UnlockBits(bitmapData2);
image.UnlockBits(bitmapData1);
return returnMap;
}//end processImageThe PixelFormat is crucial. This format determines the layout of the bitmap data. This is why the literal 4 is hard-coded (actually, it is hard-coded because I am lazy). If you use a different PixelFormat, you will need to determine the layout and offset for those formats.
Collapse Copy CodeBitmapData bitmapData1 = image.LockBits(new Rectangle(0, 0,
image.Width, image.Height),
ImageLockMode.ReadOnly,
PixelFormat.Format32bppArgb);
BitmapData bitmapData2 = returnMap.LockBits(new Rectangle(0, 0,
returnMap.Width, returnMap.Height),
ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);This code locks the bitmap and returns its image data. Since I am converting the entire image to grayscale, I lock the entire bitmap. It is possible to lock a smaller area.
Collapse Copy Codebyte* imagePointer1 = (byte*)bitmapData1.Scan0;
byte* imagePointer2 = (byte*)bitmapData2.Scan0;Pointer arithmetic is unsafe in C# :(. Too bad because it is fast! Although, it is strange getting Access Violation errors in C#. If you change the code, test thoroughly using Mathematics to make sure you don't overwrite any protected memory. Scan0 is the pointer to the first byte of the bitmap's data.
Collapse Copy Codefor(int i = 0; i < bitmapData1.Height; i++) {
for(int j = 0; j < bitmapData1.Width; j++) {The astute will immediately recognize this will only work if the bitmaps are the same size. And fortunately, this always is since this method will create the returnMap bitmap based on the input bitmap. An incorrect loop will also cause an Access Violation.
Collapse Copy CodeimagePointer2[0] = (byte)a; //Array index 0 is blue
imagePointer2[1] = (byte)a; //Array index 1 is green
imagePointer2[2] = (byte)a; //Array index 2 is red
imagePointer2[3] = imagePointer1[3]; //Array index 3 is alphaSee the comments in the code snippet above, to understand the layout of the bitmap data.
Collapse Copy Codea = (imagePointer1[0] + imagePointer1[1] + imagePointer1[2])/3;Average the three color components in the original bitmap. Keep the alpha the same, otherwise your grayscale will also cause undesired blending.
Collapse Copy CodeimagePointer1 += 4;
imagePointer2 += 4;Move forward 4 bytes:
Collapse Copy CodeimagePointer1 += bitmapData1.Stride - (bitmapData1.Width * 4);
imagePointer2 += bitmapData1.Stride - (bitmapData1.Width * 4);This is definitely the most confusing part. See this for a picture of what is happening. The width of a bitmap is actually the composed width and an unused buffer to pad the bitmap to be a multiple of 4. Width + buffer = stride. It works, so good enough for me.
Collapse Copy CodereturnMap.UnlockBits(bitmapData2);
image.UnlockBits(bitmapData1);
return returnMap;UnlockBits unlocks the bitmap data, probably a good thing to do.
Points of Interest
A good place to find answers to hard problems: Google.
People much smarter than me: MIT Cog Project.
Possible Errors
I use the bitmap data from bitmapData1 for both bitmapData1 and bitmapData2. This could lead to unexpected errors if the data were some how different. However, in testing, everything works fine. A lot more prudent coder would verify a lot of things before putting this sort of code into anything critical.
If there are any more errors, I am quite sure that the overly critical among you will gladly point them out as glaring criticisms.
License
This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.
A list of licenses authors might use can be found here
About the Author
Sunday, May 2, 2010
Area Based Motion Detection- Boss Watcher
This piece of application is a union of CodeProject articles:
Motion detection Algorithms Andrew Kirillov
Owner Drawn Resizable control Ray Cassick
Thanks to the authors of these articles. Below is what this app does:
Introduction
Working in artificial lighted, lifted ceiling, ice air conditioned IT office departments gives me a creepy feeling of being manipulated and watched from behind. Any way one should support it behind. That is why I tried to write this application. But it can also be used for alerting when there is a motion in a special area.
Background
App uses the optimized motion detection algorithm by Andrew to alert (beep), jumps pixels defined by the user.
Usage
Put your camera where suitable and by green rectangle, point where any motion will be rejected for measure, also highly sensitive area by red rectangle. App beeps when overall motion is higher than the sensitivity.
Advices
Change the below lines if you want to do something different in MotionDetector3Optimized.cs:
Collapse Copy Codeif (MainForm.vulRatio>0 && (diffCounter>(height*width/(100/MainForm.vulRatio ))))
{
System.Media.SystemSounds.Beep.Play() ;
}Change the below lines if you want to process only sensitive area in MotionDetector3Optimized.cs:
Collapse Copy Codeif ((j > ns1 && j < ns2) && (i > ns3 && i < ns4))
{
currentFrameDilatated[k] = 0;
}How to do it is your puzzle. ;)
History
30th March, 2007: Initial post
Motion detection Algorithms Andrew Kirillov
Owner Drawn Resizable control Ray Cassick
Thanks to the authors of these articles. Below is what this app does:
Introduction
Working in artificial lighted, lifted ceiling, ice air conditioned IT office departments gives me a creepy feeling of being manipulated and watched from behind. Any way one should support it behind. That is why I tried to write this application. But it can also be used for alerting when there is a motion in a special area.
Background
App uses the optimized motion detection algorithm by Andrew to alert (beep), jumps pixels defined by the user.
Usage
Put your camera where suitable and by green rectangle, point where any motion will be rejected for measure, also highly sensitive area by red rectangle. App beeps when overall motion is higher than the sensitivity.
Advices
Change the below lines if you want to do something different in MotionDetector3Optimized.cs:
Collapse Copy Codeif (MainForm.vulRatio>0 && (diffCounter>(height*width/(100/MainForm.vulRatio ))))
{
System.Media.SystemSounds.Beep.Play() ;
}Change the below lines if you want to process only sensitive area in MotionDetector3Optimized.cs:
Collapse Copy Codeif ((j > ns1 && j < ns2) && (i > ns3 && i < ns4))
{
currentFrameDilatated[k] = 0;
}How to do it is your puzzle. ;)
History
30th March, 2007: Initial post
How to display Flash animation in Visual C++ MFC application
Tutorial: How to display Flash animation in Visual C++ MFC application
You can download the source code of this example here: vc_how_to_display_flash.zip
On this page you will find step-by-step tutorial on how to display flash (SWF) animation file in your Visual C++ application. We will make VC++ application that will display Shapes.swf flash animation previously generated using our SWF Scout library (click here for "Shapes" example).
Start Visual C++ 6.0 wizard, select type and name of new MFC application project:
Click OK to start MFC Application Wizard:
Select Dialog based project type and click Next. Next page will appear:
Click Finish to create new project. Visual C++ IDE will display summary information for new project:
Click OK to open new dialog based project in VC++ IDE:
Now we should import Shockwave ActiveX control to controls toolbar. Go to Project menu, select Add To Project sub-menu and then click Components and Controls... command:
Visual C++ will display "Components and Controls Gallery" dialog:
Click on "Registered ActiveX Controls" to display list of available ActiveX controls:
Find and select Shockwave Flash Object ActiveX control in the list of available controls and click "Insert" to add this control to controls toolbar.
VC will generate special C++ class that will help to interact with this control:
Click OK and VC will generate new "CShockwaveFlash" class in ShockwaveFlash.cpp and ShockwaveFlash.h files.
"Shockwave Flash Object" control will appear on controls toolbar:
Click on "Shockwave Flash Object" icon and then click on dialog form. Answer "OK" to add new component to dialog:
New control will be added as shown below :
Now we will add dialog class member that will handle Flash control. Right-click on dialog and select "Class Wizard" to invoke Class Wizard:
Class Wizard dialog will appear. Select Member Variables tab and click Add Variable... button to add new variable:
Enter name of new varaible: m_FlashPlayer and click OK:
New click OK to close MFC ClassWizard:
Now we will place code that will load "Shapes.swf" into flash player control.
Right-click on dialog form and select "Events.." to invoke Events dialog:
Select and double-click on WM_INITDIALOG caption as shown on screenshot below:
Code Editor window will appear. CDisplayFlashDlg::OnInitDialog() function is responsible for handling initialization code so we will add our code here.
Add this line :
m_FlashPlayer.SetMovie("c:\shapes.swf");
as shown on screenshot below:
And run application by pressing F5 or using Build menu:
You will see application that will display Shapes.SWF flash movie using Shockwave Flash Object ActiveX control:
You can download the source code of this example here: vc_how_to_display_flash.zip
[an error occurred while processing this directive]
You can download the source code of this example here: vc_how_to_display_flash.zip
On this page you will find step-by-step tutorial on how to display flash (SWF) animation file in your Visual C++ application. We will make VC++ application that will display Shapes.swf flash animation previously generated using our SWF Scout library (click here for "Shapes" example).
Start Visual C++ 6.0 wizard, select type and name of new MFC application project:
Click OK to start MFC Application Wizard:
Select Dialog based project type and click Next. Next page will appear:
Click Finish to create new project. Visual C++ IDE will display summary information for new project:
Click OK to open new dialog based project in VC++ IDE:
Now we should import Shockwave ActiveX control to controls toolbar. Go to Project menu, select Add To Project sub-menu and then click Components and Controls... command:
Visual C++ will display "Components and Controls Gallery" dialog:
Click on "Registered ActiveX Controls" to display list of available ActiveX controls:
Find and select Shockwave Flash Object ActiveX control in the list of available controls and click "Insert" to add this control to controls toolbar.
VC will generate special C++ class that will help to interact with this control:
Click OK and VC will generate new "CShockwaveFlash" class in ShockwaveFlash.cpp and ShockwaveFlash.h files.
"Shockwave Flash Object" control will appear on controls toolbar:
Click on "Shockwave Flash Object" icon and then click on dialog form. Answer "OK" to add new component to dialog:
New control will be added as shown below :
Now we will add dialog class member that will handle Flash control. Right-click on dialog and select "Class Wizard" to invoke Class Wizard:
Class Wizard dialog will appear. Select Member Variables tab and click Add Variable... button to add new variable:
Enter name of new varaible: m_FlashPlayer and click OK:
New click OK to close MFC ClassWizard:
Now we will place code that will load "Shapes.swf" into flash player control.
Right-click on dialog form and select "Events.." to invoke Events dialog:
Select and double-click on WM_INITDIALOG caption as shown on screenshot below:
Code Editor window will appear. CDisplayFlashDlg::OnInitDialog() function is responsible for handling initialization code so we will add our code here.
Add this line :
m_FlashPlayer.SetMovie("c:\shapes.swf");
as shown on screenshot below:
And run application by pressing F5 or using Build menu:
You will see application that will display Shapes.SWF flash movie using Shockwave Flash Object ActiveX control:
You can download the source code of this example here: vc_how_to_display_flash.zip
[an error occurred while processing this directive]
Add COM and ActiveX Support in .NET Compact Framework Using Odyssey Software CFCOM
.NET Compact Framework
Add COM and ActiveX Support in .NET Compact Framework Using Odyssey Software CFCOM
Andy Sjöström
businessanyplace.net
September 2003
Applies to:
Microsoft® .NET Compact Framework version 1.0
Microsoft Visual Studio® .NET 2003
Odyssey Software CFCOM version 1.0
Windows Mobile™ 2003 software for Pocket PCs
Summary:Learn how to add COM and ActiveX support to .NET Compact Framework applications in Visual C#. Also, see how the Macromedia Flash Player 6 for Windows Mobile 2003 software for Pocket PCs can be used in .NET Compact Framework applications. (11 printed pages)
Background:COM and ActiveX® interoperability is available in the full .NET Framework. Managed .NET Compact Framework code can call unmanaged functions residing in DLLs using Platform Invoke (P/Invoke), but full COM and ActiveX support is not available in the .NET Compact Framework. Odyssey Software's CFCOM becomes the intervening layer that exposes COM objects and ActiveX controls as .NET equivalences to .NET Compact Framework applications using a memory foot print that is only about 30K.
Odyssey Software CFCOM enables transparent access to controls such as the Windows Media Player and objects such as the Pocket Outlook Object Model and ADOCE, and virtually any third party COM or ActiveX component. CFCOM can be licensed from Odyssey Software, although the code sample in this article will work with the license key found in the source code.
Download cfcom.exe from the Microsoft Download Center.
Contents
Introduction
Odyssey Software CFCOM
Code Walk-through
Macromedia Flash 6 Player for Pocket PC
Flash Anyplace
Flash Anyplace Code Walk-through
Conclusion
Introduction
The Microsoft#&174; .NET Compact Framework is a hierarchy of namespaces and classes that, despite its small size, is very similar to the full .NET framework. It is natural that some features in the full .NET Framework had to be left out for size and performance reasons. One of these features is COM interoperability. Many core elements of Microsoft Windows® CE and the Pocket PC within connectivity, messaging, communication, graphics, media, data storage, and personal information management (PIM) are implemented as COM objects or ActiveX controls. There are also many third party COM and ActiveX controls that are intended for use with mobile devices, for example within GPS, charting, security and connectivity. Platform Invoke (P/Invoke) found in the System.Runtime.InteropServices namespace makes it possible to call simple unmanaged code functions. One example of a third-party utilities provider with which existing .NET Compact Framework interoperability services are sufficient is Sapphire Solutions Ltd (http://www.sapphire-solutions.co.uk/). Sapphire Solutions has components and source code for remote access services, encryption, and so on. Sapphire Solutions components are shipped with .NET Compact Framework code samples utilizing Platform Invoke, which makes it easier to start using the code.
Calling more complex unmanaged functions requires a significant amount of complex coding that involves more challenging areas, such as marshalling. Developing COM wrappers often results in simplified function-based interfaces. Integrating COM and ActiveX into managed code is challenging.
Odyssey Software CFCOM
Odyssey Software is not a stranger in the Windows CE developer community. They have been around since 1996 and have played an important role in enabling enterprise mobile application development through products such as CEfusion and ViaXML. The company continues to provide tools to the mobile application development community, this time with an intervening layer between COM and the Compact Framework that exposes COM types as .NET equivalents. CFCOM provides comprehensive COM and ActiveX interoperability to Compact Framework applications.
CFCOM can be seen as a wrapper that enables the use of both COM and .NET objects in the same application. A Compact Framework object is controlled by the Common Language Runtime (CLR) in managed memory while a COM object is held in unmanaged memory. CFCOM is implemented to act as a transparent wrapper, making interaction of managed and unmanaged components possible: setting and getting property values, calling methods, capturing COM events, handling advanced type conversion functionality (a k a marshalling) and generating native .NET exceptions from exceptions occurring in unmanaged code.
To the developer, CFCOM is fundamentally composed of three files:
Odyssey.CFCOM.dll contains the CFCOM Runtime.
Odyssey.CFCOM.Design.dll is required for design-time support in Microsoft Visual Studio®. This file is placed, by the installer, in the directory C:\Program Files\Microsoft Visual Studio .NET 2003\CompactFrameworkSDK\ v1.0.5000\Windows CE\Designer.
Odyssey.CFCOM.1248.dll is the native (device-specific) runtime component.
Figure 1 illustrates the ComObject and ComControl tools added to the toolbox.
Figure 1. ComObject and ComControl tools added to the toolbox
Odyssey Software also includes a sample Windows Media Player Wrapper Designer that the developer can add to the toolbox and set property values in design-time. Compact Framework applications using Windows Media Player can then quickly be created.
For eMbedded Visual Basic developers, CFCOM means being able to carry on using familiar COM objects and ActiveX controls, without the need to learn C/C++ and P/Invoke. eMbedded Visual C++ developers can continue to write the performance-critical portions of their applications in C++, without having to create native wrappers and dealing with custom marshalling.
For more information on Odyssey Software CFCOM, you can read the Microsoft Case Study, COM Interop Library Gives Compact Framework Applications Easy Access to COM Components and ActiveX Controls.
Code Walk-through
CFCOM is designed to be as transparent as possible. There are only two things that need to be set before using an ActiveX control or COM object:
The ProgID or CLSID of the ActiveX control or COM object
The CFCOM license key which is acquired from Odyssey Software
After this, it is possible to start using the component using CFCOM's Invoke method. The following code shows how the Windows Media Player can be used in a Compact Framework application.
The license key is set in the application's main entry point.
CopyComObject.AddLicense(CFCOM_LICENSEKEY);
The one-form application can then start using the Windows Media Player.
public class frmMain : System.Windows.Forms.Form
Copy{
private const string CFCOM_LICENSEKEY = " XXXXXXXXXXXXXXXXXXXXXXXXXX ";
public frmMain()
{
InitializeComponent();
mpc.ProgID = "WMPCEOCX.WMP";
mpc.Invoke("ShowControls", InvokeFlags.SetProperty, false);
mpc.Invoke("ShowAudioControls", InvokeFlags.SetProperty, false);
mpc.Invoke("ShowPositionControls", InvokeFlags.SetProperty, false);
mpc.Invoke("ShowTracker", InvokeFlags.SetProperty, false);
mpc.Invoke("ShowStatusBar", InvokeFlags.SetProperty, true);
mpc.Invoke("Autostart", InvokeFlags.SetProperty, true);
mpc.Invoke("Filename", InvokeFlags.SetProperty, "http://news.com.com/1604-2-966406-1.asx?msft_awe+win");
mpc.Invoke("SendMouseClickEvents", InvokeFlags.SetProperty, true);
mpc.ComObject.ComEvent += new ComEventHandler(mpc_COMEvent);
}
Note how the ProgID is set and then how the Invoke method is used to set properties. The InvokeFlags enumeration includes SetProperty, GetProperty, and CallMethod.
For more information about this code and how to add media to Compact Framework applications, see the article, "Play Media with .NET CF using CFCOM" located at: http://www.businessanyplace.net/?p=cfcommedia.
Figure 2 illustrates the application running with the Windows Media Player ActiveX control.
Figure 2. Windows Media Player Active X control
Macromedia Flash 6 Player for Pocket PC
Macromedia has made available a free Pocket PC player for Macromedia Flash content. Flash is a popular platform for delivering high profile content, movies and even mobile applications. The Flash development community is active and growing. Developers using Flash in mobile application development based on Visual Studio .NET and the Compact Framework can easily bring out fantastic content and applications. The Pocket PC player can be downloaded for personal use free from the Macromedia Web site at: http://www.macromedia.com/software/flashplayer/pocketpc/2002.html.
The player is an ActiveX control which most commonly is hosted in Pocket Internet Explorer and is physically installed as flash.dll in the directory \Windows\Macromedia. Once the player has been installed the user can view Macromedia Flash content in Pocket Internet Explorer. One way to do that is to put the Flash content file (.swf) and HTML file in the same directory and open the HTML file. The HTML file reveals how the ActiveX control is called. The following snippet is from such an HTML file:
Copy
book
codebase="http://active.macromedia.com/flash2/cabs/swflash.cab#version=6,0,0,0"
ID=rapier4 WIDTH=230 HEIGHT=255>
The ActiveX control is called using the parameters mentioned after the "PARAM" properties. The movie to be played is called "book.swf," the Flash context menu is turned off, quality is high and the background color is set to #53695C.
Figure 3 illustrates the result in Pocket Internet Explorer.
Figure 3. Pocket PC Flash player in action
The Macromedia Flash content shown in this sample is designed by the Flash developer Phillip Torrone from flashenabled.com (http://www.flashenabled.com/). More samples like this and more Pocket PC and Macromedia Flash related information can be found on the site Pocket PC Flash (http://www.pocketpcflash.net/home.htm).
For more information on Flash player parameters, see the article, "Scripting with Flash Player and ActiveX" on the Macromedia Web site at: http://www.macromedia.com/support/flash/ts/documents/activex_script.htm.
Let's take a look at how to put the Flash player to work in a Compact Framework application.
Flash Anyplace
Flash Anyplace is an application that hosts the Flash ActiveX control using CFCOM. Flash Anyplace can play back content and provide the user with some control of how the content is played back. Figure 4 illustrates the first form.
Figure 4. Flash player settings
In the form, the user can set what Flash file to view and control the way the Flash file should behave. The second form (figure 5) illustrates the actual Flash file being played.
Figure 5. Viewing the file
Flash Anyplace Code Walk-through
The project consists of two forms. The following code executes when the user taps the "Play" button on the first form:
Copyprivate void btnPlay_Click(object sender, System.EventArgs e)
{
Cursor.Current = Cursors.WaitCursor;
frmFlashAnyplacePlayer flashAnyplacePlayer = new frmFlashAnyplacePlayer(this, this.txtFlashFile.Text, this.cmbQuality.SelectedIndex, this.cmbScale.SelectedIndex, this.chkLoop.Checked, this.chkMenu.Checked, this.txtBackgroundColor.Text);
flashAnyplacePlayer.ShowDialog();
}
The parameters are sent to the second form, which has a CFCOM ComControl on it. The ProgID of Macromedia's Flash Player is ShockwaveFlash.ShockwaveFlash. This is set in the form's InitializeComponent section:
Copythis.flashPlayer = new Odyssey.CFCOM.ComControl();
this.flashPlayer.ProgID = "ShockwaveFlash.ShockwaveFlash";
It is also possible to use the CLSID found in the HTML file of Flash content played in Pocket Internet Explorer. The line would then be:
Copythis.flashPlayer.ProgID = "{D27CDB6E-AE6D-11cf-96B8-444553540000}";
Note that the license key has to be set before interacting with the ComControl. First, set the license key string in the declarations:
Copyprivate const string CFCOM_LICENSEKEY = "XUHXRATLAUVKL96EFVBH9BEUV7";
This license key enables usage of CFCOM with the Flash player only (using the ProgID) and with this project only. The AddLicense method should be placed just prior to the Application.Run location in the startup form. CFCOM tracks the license on a per process basis.
Copystatic void Main()
{
// Set license key
ComControl.AddLicense(CFCOM_LICENSEKEY);
Application.Run(new frmFlashAnyplace());
}
If the location of the Flash file is on the Internet, use the URL as the movie location. For example, http://www.businessanyplace.net/files/FlashAnyplace.swf. If the location of the Flash file is locally on the Pocket PC use the following format:
Copyfile://\My%20Documents\mymovie.swf
The Flash Anyplace project has a Flash file as a Content item, which is installed in the application directory. To get the current application directory and suggest the Flash file as default, the following code is executed:
Copy// Get application directory
string currentDir = new FileInfo(Assembly.GetExecutingAssembly().GetName().CodeBase).DirectoryName;
// Set default values
txtFlashFile.Text = @"file://" + currentDir + @"\book.swf";
The load event of the second form passes the parameters to the Flash player through CFCOM:
Copytry
{
flashPlayer.Invoke("quality", InvokeFlags.SetProperty, quality);
flashPlayer.Invoke("bgcolor", InvokeFlags.SetProperty, backgroundColor);
flashPlayer.Invoke("scale", InvokeFlags.SetProperty, scale);
flashPlayer.Invoke("loop", InvokeFlags.SetProperty, loop);
flashPlayer.Invoke("menu", InvokeFlags.SetProperty, menu);
flashPlayer.Invoke("movie", InvokeFlags.SetProperty, flashFile);
}
catch(Exception ex)
{
MessageBox.Show(ex.Message);
}
finally
{
parentForm.Hide();
Cursor.Current = Cursors.Default;
}
If we had tried to pass an invalid parameter name to the Macromedia Flash Player, the exception handling would have returned an appropriate exception. For example, when passing "movie1" as a property instead of "movie", CFCOM would have thrown the exception "0x80020006: DISP_E_UNKNOWNNAME."
Finally, when the user leaves the form with the Flash player, the CFCOM ComControl is disposed:
Copyprivate void frmFlashAnyplacePlayer_Closing(object sender, System.ComponentModel.CancelEventArgs e)
{
flashPlayer.Dispose();
parentForm.Show();
}
From CFCOM's help file: "As straightforward as CFCOM may be, there is one very important issue to be aware of. Due to the non-deterministic nature of garbage collection, it is frequently necessary to dispose of COM resources manually. This is accomplished with the ComObject.Dispose and ComControl.Dispose methods. Certain ActiveX controls, such as Windows Media Player, exhibit strange behavior when they are not disposed of before garbage collection."
Conclusion
Odyssey Software CFCOM addresses some very challenging areas of Compact Framework development: ActiveX and COM interoperability. By implementing a transparent layer between managed and unmanaged code, the developer can stay focused on the core challenges that are bringing the right value to its users.
Add COM and ActiveX Support in .NET Compact Framework Using Odyssey Software CFCOM
Andy Sjöström
businessanyplace.net
September 2003
Applies to:
Microsoft® .NET Compact Framework version 1.0
Microsoft Visual Studio® .NET 2003
Odyssey Software CFCOM version 1.0
Windows Mobile™ 2003 software for Pocket PCs
Summary:Learn how to add COM and ActiveX support to .NET Compact Framework applications in Visual C#. Also, see how the Macromedia Flash Player 6 for Windows Mobile 2003 software for Pocket PCs can be used in .NET Compact Framework applications. (11 printed pages)
Background:COM and ActiveX® interoperability is available in the full .NET Framework. Managed .NET Compact Framework code can call unmanaged functions residing in DLLs using Platform Invoke (P/Invoke), but full COM and ActiveX support is not available in the .NET Compact Framework. Odyssey Software's CFCOM becomes the intervening layer that exposes COM objects and ActiveX controls as .NET equivalences to .NET Compact Framework applications using a memory foot print that is only about 30K.
Odyssey Software CFCOM enables transparent access to controls such as the Windows Media Player and objects such as the Pocket Outlook Object Model and ADOCE, and virtually any third party COM or ActiveX component. CFCOM can be licensed from Odyssey Software, although the code sample in this article will work with the license key found in the source code.
Download cfcom.exe from the Microsoft Download Center.
Contents
Introduction
Odyssey Software CFCOM
Code Walk-through
Macromedia Flash 6 Player for Pocket PC
Flash Anyplace
Flash Anyplace Code Walk-through
Conclusion
Introduction
The Microsoft#&174; .NET Compact Framework is a hierarchy of namespaces and classes that, despite its small size, is very similar to the full .NET framework. It is natural that some features in the full .NET Framework had to be left out for size and performance reasons. One of these features is COM interoperability. Many core elements of Microsoft Windows® CE and the Pocket PC within connectivity, messaging, communication, graphics, media, data storage, and personal information management (PIM) are implemented as COM objects or ActiveX controls. There are also many third party COM and ActiveX controls that are intended for use with mobile devices, for example within GPS, charting, security and connectivity. Platform Invoke (P/Invoke) found in the System.Runtime.InteropServices namespace makes it possible to call simple unmanaged code functions. One example of a third-party utilities provider with which existing .NET Compact Framework interoperability services are sufficient is Sapphire Solutions Ltd (http://www.sapphire-solutions.co.uk/). Sapphire Solutions has components and source code for remote access services, encryption, and so on. Sapphire Solutions components are shipped with .NET Compact Framework code samples utilizing Platform Invoke, which makes it easier to start using the code.
Calling more complex unmanaged functions requires a significant amount of complex coding that involves more challenging areas, such as marshalling. Developing COM wrappers often results in simplified function-based interfaces. Integrating COM and ActiveX into managed code is challenging.
Odyssey Software CFCOM
Odyssey Software is not a stranger in the Windows CE developer community. They have been around since 1996 and have played an important role in enabling enterprise mobile application development through products such as CEfusion and ViaXML. The company continues to provide tools to the mobile application development community, this time with an intervening layer between COM and the Compact Framework that exposes COM types as .NET equivalents. CFCOM provides comprehensive COM and ActiveX interoperability to Compact Framework applications.
CFCOM can be seen as a wrapper that enables the use of both COM and .NET objects in the same application. A Compact Framework object is controlled by the Common Language Runtime (CLR) in managed memory while a COM object is held in unmanaged memory. CFCOM is implemented to act as a transparent wrapper, making interaction of managed and unmanaged components possible: setting and getting property values, calling methods, capturing COM events, handling advanced type conversion functionality (a k a marshalling) and generating native .NET exceptions from exceptions occurring in unmanaged code.
To the developer, CFCOM is fundamentally composed of three files:
Odyssey.CFCOM.dll contains the CFCOM Runtime.
Odyssey.CFCOM.Design.dll is required for design-time support in Microsoft Visual Studio®. This file is placed, by the installer, in the directory C:\Program Files\Microsoft Visual Studio .NET 2003\CompactFrameworkSDK\ v1.0.5000\Windows CE\Designer.
Odyssey.CFCOM.1248.dll is the native (device-specific) runtime component.
Figure 1 illustrates the ComObject and ComControl tools added to the toolbox.
Figure 1. ComObject and ComControl tools added to the toolbox
Odyssey Software also includes a sample Windows Media Player Wrapper Designer that the developer can add to the toolbox and set property values in design-time. Compact Framework applications using Windows Media Player can then quickly be created.
For eMbedded Visual Basic developers, CFCOM means being able to carry on using familiar COM objects and ActiveX controls, without the need to learn C/C++ and P/Invoke. eMbedded Visual C++ developers can continue to write the performance-critical portions of their applications in C++, without having to create native wrappers and dealing with custom marshalling.
For more information on Odyssey Software CFCOM, you can read the Microsoft Case Study, COM Interop Library Gives Compact Framework Applications Easy Access to COM Components and ActiveX Controls.
Code Walk-through
CFCOM is designed to be as transparent as possible. There are only two things that need to be set before using an ActiveX control or COM object:
The ProgID or CLSID of the ActiveX control or COM object
The CFCOM license key which is acquired from Odyssey Software
After this, it is possible to start using the component using CFCOM's Invoke method. The following code shows how the Windows Media Player can be used in a Compact Framework application.
The license key is set in the application's main entry point.
CopyComObject.AddLicense(CFCOM_LICENSEKEY);
The one-form application can then start using the Windows Media Player.
public class frmMain : System.Windows.Forms.Form
Copy{
private const string CFCOM_LICENSEKEY = " XXXXXXXXXXXXXXXXXXXXXXXXXX ";
public frmMain()
{
InitializeComponent();
mpc.ProgID = "WMPCEOCX.WMP";
mpc.Invoke("ShowControls", InvokeFlags.SetProperty, false);
mpc.Invoke("ShowAudioControls", InvokeFlags.SetProperty, false);
mpc.Invoke("ShowPositionControls", InvokeFlags.SetProperty, false);
mpc.Invoke("ShowTracker", InvokeFlags.SetProperty, false);
mpc.Invoke("ShowStatusBar", InvokeFlags.SetProperty, true);
mpc.Invoke("Autostart", InvokeFlags.SetProperty, true);
mpc.Invoke("Filename", InvokeFlags.SetProperty, "http://news.com.com/1604-2-966406-1.asx?msft_awe+win");
mpc.Invoke("SendMouseClickEvents", InvokeFlags.SetProperty, true);
mpc.ComObject.ComEvent += new ComEventHandler(mpc_COMEvent);
}
Note how the ProgID is set and then how the Invoke method is used to set properties. The InvokeFlags enumeration includes SetProperty, GetProperty, and CallMethod.
For more information about this code and how to add media to Compact Framework applications, see the article, "Play Media with .NET CF using CFCOM" located at: http://www.businessanyplace.net/?p=cfcommedia.
Figure 2 illustrates the application running with the Windows Media Player ActiveX control.
Figure 2. Windows Media Player Active X control
Macromedia Flash 6 Player for Pocket PC
Macromedia has made available a free Pocket PC player for Macromedia Flash content. Flash is a popular platform for delivering high profile content, movies and even mobile applications. The Flash development community is active and growing. Developers using Flash in mobile application development based on Visual Studio .NET and the Compact Framework can easily bring out fantastic content and applications. The Pocket PC player can be downloaded for personal use free from the Macromedia Web site at: http://www.macromedia.com/software/flashplayer/pocketpc/2002.html.
The player is an ActiveX control which most commonly is hosted in Pocket Internet Explorer and is physically installed as flash.dll in the directory \Windows\Macromedia. Once the player has been installed the user can view Macromedia Flash content in Pocket Internet Explorer. One way to do that is to put the Flash content file (.swf) and HTML file in the same directory and open the HTML file. The HTML file reveals how the ActiveX control is called. The following snippet is from such an HTML file:
Copy
codebase="http://active.macromedia.com/flash2/cabs/swflash.cab#version=6,0,0,0"
ID=rapier4 WIDTH=230 HEIGHT=255>
The ActiveX control is called using the parameters mentioned after the "PARAM" properties. The movie to be played is called "book.swf," the Flash context menu is turned off, quality is high and the background color is set to #53695C.
Figure 3 illustrates the result in Pocket Internet Explorer.
Figure 3. Pocket PC Flash player in action
The Macromedia Flash content shown in this sample is designed by the Flash developer Phillip Torrone from flashenabled.com (http://www.flashenabled.com/). More samples like this and more Pocket PC and Macromedia Flash related information can be found on the site Pocket PC Flash (http://www.pocketpcflash.net/home.htm).
For more information on Flash player parameters, see the article, "Scripting with Flash Player and ActiveX" on the Macromedia Web site at: http://www.macromedia.com/support/flash/ts/documents/activex_script.htm.
Let's take a look at how to put the Flash player to work in a Compact Framework application.
Flash Anyplace
Flash Anyplace is an application that hosts the Flash ActiveX control using CFCOM. Flash Anyplace can play back content and provide the user with some control of how the content is played back. Figure 4 illustrates the first form.
Figure 4. Flash player settings
In the form, the user can set what Flash file to view and control the way the Flash file should behave. The second form (figure 5) illustrates the actual Flash file being played.
Figure 5. Viewing the file
Flash Anyplace Code Walk-through
The project consists of two forms. The following code executes when the user taps the "Play" button on the first form:
Copyprivate void btnPlay_Click(object sender, System.EventArgs e)
{
Cursor.Current = Cursors.WaitCursor;
frmFlashAnyplacePlayer flashAnyplacePlayer = new frmFlashAnyplacePlayer(this, this.txtFlashFile.Text, this.cmbQuality.SelectedIndex, this.cmbScale.SelectedIndex, this.chkLoop.Checked, this.chkMenu.Checked, this.txtBackgroundColor.Text);
flashAnyplacePlayer.ShowDialog();
}
The parameters are sent to the second form, which has a CFCOM ComControl on it. The ProgID of Macromedia's Flash Player is ShockwaveFlash.ShockwaveFlash. This is set in the form's InitializeComponent section:
Copythis.flashPlayer = new Odyssey.CFCOM.ComControl();
this.flashPlayer.ProgID = "ShockwaveFlash.ShockwaveFlash";
It is also possible to use the CLSID found in the HTML file of Flash content played in Pocket Internet Explorer. The line would then be:
Copythis.flashPlayer.ProgID = "{D27CDB6E-AE6D-11cf-96B8-444553540000}";
Note that the license key has to be set before interacting with the ComControl. First, set the license key string in the declarations:
Copyprivate const string CFCOM_LICENSEKEY = "XUHXRATLAUVKL96EFVBH9BEUV7";
This license key enables usage of CFCOM with the Flash player only (using the ProgID) and with this project only. The AddLicense method should be placed just prior to the Application.Run location in the startup form. CFCOM tracks the license on a per process basis.
Copystatic void Main()
{
// Set license key
ComControl.AddLicense(CFCOM_LICENSEKEY);
Application.Run(new frmFlashAnyplace());
}
If the location of the Flash file is on the Internet, use the URL as the movie location. For example, http://www.businessanyplace.net/files/FlashAnyplace.swf. If the location of the Flash file is locally on the Pocket PC use the following format:
Copyfile://\My%20Documents\mymovie.swf
The Flash Anyplace project has a Flash file as a Content item, which is installed in the application directory. To get the current application directory and suggest the Flash file as default, the following code is executed:
Copy// Get application directory
string currentDir = new FileInfo(Assembly.GetExecutingAssembly().GetName().CodeBase).DirectoryName;
// Set default values
txtFlashFile.Text = @"file://" + currentDir + @"\book.swf";
The load event of the second form passes the parameters to the Flash player through CFCOM:
Copytry
{
flashPlayer.Invoke("quality", InvokeFlags.SetProperty, quality);
flashPlayer.Invoke("bgcolor", InvokeFlags.SetProperty, backgroundColor);
flashPlayer.Invoke("scale", InvokeFlags.SetProperty, scale);
flashPlayer.Invoke("loop", InvokeFlags.SetProperty, loop);
flashPlayer.Invoke("menu", InvokeFlags.SetProperty, menu);
flashPlayer.Invoke("movie", InvokeFlags.SetProperty, flashFile);
}
catch(Exception ex)
{
MessageBox.Show(ex.Message);
}
finally
{
parentForm.Hide();
Cursor.Current = Cursors.Default;
}
If we had tried to pass an invalid parameter name to the Macromedia Flash Player, the exception handling would have returned an appropriate exception. For example, when passing "movie1" as a property instead of "movie", CFCOM would have thrown the exception "0x80020006: DISP_E_UNKNOWNNAME."
Finally, when the user leaves the form with the Flash player, the CFCOM ComControl is disposed:
Copyprivate void frmFlashAnyplacePlayer_Closing(object sender, System.ComponentModel.CancelEventArgs e)
{
flashPlayer.Dispose();
parentForm.Show();
}
From CFCOM's help file: "As straightforward as CFCOM may be, there is one very important issue to be aware of. Due to the non-deterministic nature of garbage collection, it is frequently necessary to dispose of COM resources manually. This is accomplished with the ComObject.Dispose and ComControl.Dispose methods. Certain ActiveX controls, such as Windows Media Player, exhibit strange behavior when they are not disposed of before garbage collection."
Conclusion
Odyssey Software CFCOM addresses some very challenging areas of Compact Framework development: ActiveX and COM interoperability. By implementing a transparent layer between managed and unmanaged code, the developer can stay focused on the core challenges that are bringing the right value to its users.
Easy Motion Detection
Introduction
When I worked on another project, I got this method to implement motion detection.
I tried some other ways to do this motion detection. But I got frustrated when I used them. Because I don't really need any advanced algorithms to do this one, I thought a little and got a nice idea to do it. And you can also do it now.
Just about 5 lines in Visual C++ .NET code and a few lines in Flash Action Script.
Background + Prerequisites
You should have pre-installed the .NET version 1.1 and Flash player 6 or higher for testing this code. They are freely available at the links given below:
.NET 1.1 redistributable package
Flash player
Using the Code
The first message comes when the application starts. The others come when it detects motion. The camera preview part and the motion detecting part are actually being done by the flash scripts. Finally the Visual C++ .NET application gets the motion via VC COM interface fscommand();
Collapse Copy Code//get the real camera & set the motion level + quality
camera = Camera.get();
camera.setMotionLevel(25, 2);
camera.setQuality(0,5);
//embedded video object
myVideoObject.hide();
//get the preview
myVideoObject.attachVideo(camera);
//call when camera detects a motion
camera.onActivity = function(mode)
{
if (mode)
{
trace("Kawru hari awa. Camerawa langata.");
//get into the Visual C++
fscommand("MOTION_DETECTED", "OK");
}
};Camera.get(); will return the camera object with the default connected interface. Then we can set the motion level and the quality. Finally attach the camera object to the Embedded Video object.
When activity happens, it gives the feed into the VC++.
Collapse Copy Codeprivate:
AxInterop::ShockwaveFlashObjects::AxShockwaveFlash* axShockwaveFlash1;
String* sFlashPath = System::Environment::CurrentDirectory;
String* sCurrentDir = sFlashPath;
//set the path of GUI
file = new String("\\detector.swf");
sFlashPath = String::Concat(sFlashPath, file);
iStartApp = 0;
//load Main detector GUI
axShockwaveFlash1->LoadMovie(0, sFlashPath);In Visual C++ .NET, just use the axShockwaveFlash COM interface to load the Main Flash GUI. Then use axShockwaveFlash1_FSCommand event to interface with the COM.
Additional Resources
Macromedia Flash Live Doc (Camera class)
License
This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)
About the Author
Nimesha Ranasinghe
Member Graduate student of Mixed Reality Lab, National University of Singapore
Occupation: Engineer
Company: Mixed Reality Lab, National University of Singapor
Location: Singapore
Other popular C++ / CLI articles:
.NET Dynamic Software Load Balancing
A Draft Implementation of an Idea for .NET Dynamic Software Load Balancing
Quake II .NET
A port of the famous C-language based game engine to Visual C++ with a .NET managed heads-up display.
A first look at C++/CLI
A brief look at the new C++/CLI syntax and how it improves over the old MC++ syntax
OLE DB - Bound controls
Shows how you can use data bound controls with OLE DB
How to get the clients IP address using TcpClient in Managed C++
How to get the clients IP address using TcpClient in Managed C++
When I worked on another project, I got this method to implement motion detection.
I tried some other ways to do this motion detection. But I got frustrated when I used them. Because I don't really need any advanced algorithms to do this one, I thought a little and got a nice idea to do it. And you can also do it now.
Just about 5 lines in Visual C++ .NET code and a few lines in Flash Action Script.
Background + Prerequisites
You should have pre-installed the .NET version 1.1 and Flash player 6 or higher for testing this code. They are freely available at the links given below:
.NET 1.1 redistributable package
Flash player
Using the Code
The first message comes when the application starts. The others come when it detects motion. The camera preview part and the motion detecting part are actually being done by the flash scripts. Finally the Visual C++ .NET application gets the motion via VC COM interface fscommand();
Collapse Copy Code//get the real camera & set the motion level + quality
camera = Camera.get();
camera.setMotionLevel(25, 2);
camera.setQuality(0,5);
//embedded video object
myVideoObject.hide();
//get the preview
myVideoObject.attachVideo(camera);
//call when camera detects a motion
camera.onActivity = function(mode)
{
if (mode)
{
trace("Kawru hari awa. Camerawa langata.");
//get into the Visual C++
fscommand("MOTION_DETECTED", "OK");
}
};Camera.get(); will return the camera object with the default connected interface. Then we can set the motion level and the quality. Finally attach the camera object to the Embedded Video object.
When activity happens, it gives the feed into the VC++.
Collapse Copy Codeprivate:
AxInterop::ShockwaveFlashObjects::AxShockwaveFlash* axShockwaveFlash1;
String* sFlashPath = System::Environment::CurrentDirectory;
String* sCurrentDir = sFlashPath;
//set the path of GUI
file = new String("\\detector.swf");
sFlashPath = String::Concat(sFlashPath, file);
iStartApp = 0;
//load Main detector GUI
axShockwaveFlash1->LoadMovie(0, sFlashPath);In Visual C++ .NET, just use the axShockwaveFlash COM interface to load the Main Flash GUI. Then use axShockwaveFlash1_FSCommand event to interface with the COM.
Additional Resources
Macromedia Flash Live Doc (Camera class)
License
This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)
About the Author
Nimesha Ranasinghe
Member Graduate student of Mixed Reality Lab, National University of Singapore
Occupation: Engineer
Company: Mixed Reality Lab, National University of Singapor
Location: Singapore
Other popular C++ / CLI articles:
.NET Dynamic Software Load Balancing
A Draft Implementation of an Idea for .NET Dynamic Software Load Balancing
Quake II .NET
A port of the famous C-language based game engine to Visual C++ with a .NET managed heads-up display.
A first look at C++/CLI
A brief look at the new C++/CLI syntax and how it improves over the old MC++ syntax
OLE DB - Bound controls
Shows how you can use data bound controls with OLE DB
How to get the clients IP address using TcpClient in Managed C++
How to get the clients IP address using TcpClient in Managed C++
Motion Detection Algorithms
Introduction
There are many approaches for motion detection in a continuous video stream. All of them are based on comparing of the current video frame with one from the previous frames or with something that we'll call background. In this article, I'll try to describe some of the most common approaches.
In description of these algorithms I'll use the AForge.NET framework, which is described in some other articles on Code Project: [1], [2]. So, if you are common with it, it will only help.
The demo application supports the following types of video sources:
AVI files (using Video for Windows, interop library is included);
updating JPEG from internet cameras;
MJPEG (motion JPEG) streams from different internet cameras;
local capture device (USB cameras or other capture devices, DirectShow interop library is included).
Algorithms
One of the most common approaches is to compare the current frame with the previous one. It's useful in video compression when you need to estimate changes and to write only the changes, not the whole frame. But it is not the best one for motion detection applications. So, let me describe the idea more closely.
Assume that we have an original 24 bpp RGB image called current frame (image), a grayscale copy of it (currentFrame) and previous video frame also gray scaled (backgroundFrame). First of all, let's find the regions where these two frames are differing a bit. For the purpose we can use Difference and Threshold filters.
Collapse Copy Code// create filters
Difference differenceFilter = new Difference( );
IFilter thresholdFilter = new Threshold( 15 );
// set backgroud frame as an overlay for difference filter
differenceFilter.OverlayImage = backgroundFrame;
// apply the filters
Bitmap tmp1 = differenceFilter.Apply( currentFrame );
Bitmap tmp2 = thresholdFilter.Apply( tmp1 );On this step we'll get an image with white pixels on the place where the current frame is different from the previous frame on the specified threshold value. It's already possible to count the pixels, and if the amount of it will be greater than a predefined alarm level we can signal about a motion event.
But most cameras produce a noisy image, so we'll get motion in such places, where there is no motion at all. To remove random noisy pixels, we can use an Erosion filter, for example. So, we'll get now mostly only the regions where the actual motion was.
Collapse Copy Code// create filter
IFilter erosionFilter = new Erosion( );
// apply the filter
Bitmap tmp3 = erosionFilter.Apply( tmp2 );The simplest motion detector is ready! We can highlight the motion regions if needed.
Collapse Copy Code// extract red channel from the original image
IFilter extrachChannel = new ExtractChannel( RGB.R );
Bitmap redChannel = extrachChannel.Apply( image );
// merge red channel with motion regions
Merge mergeFilter = new Merge( );
mergeFilter.OverlayImage = tmp3;
Bitmap tmp4 = mergeFilter.Apply( redChannel );
// replace red channel in the original image
ReplaceChannel replaceChannel = new ReplaceChannel( RGB.R );
replaceChannel.ChannelImage = tmp4;
Bitmap tmp5 = replaceChannel.Apply( image );Here is the result of it:
From the above picture we can see the disadvantages of the approach. If the object is moving smoothly we'll receive small changes from frame to frame. So, it's impossible to get the whole moving object. Things become worse, when the object is moving so slowly, when the algorithms will not give any result at all.
There is another approach. It's possible to compare the current frame not with the previous one but with the first frame in the video sequence. So, if there were no objects in the initial frame, comparison of the current frame with the first one will give us the whole moving object independently of its motion speed. But, the approach has a big disadvantage - what will happen, if there was, for example, a car on the first frame, but then it is gone? Yes, we'll always have motion detected on the place, where the car was. Of course, we can renew the initial frame sometimes, but still it will not give us good results in the cases where we can not guarantee that the first frame will contain only static background. But, there can be an inverse situation. If I'll put a picture on the wall in the room? I'll get motion detected until the initial frame will be renewed.
The most efficient algorithms are based on building the so called background of the scene and comparing each current frame with the background. There are many approaches to build the scene, but most of them are too complex. I'll describe here my approach for building the background. It's rather simple and can be realized very quickly.
As in the previous case, let's assume that we have an original 24 bpp RGB image called current frame (image), a grayscale copy of it (currentFrame) and a background frame also gray scaled (backgroundFrame). At the beginning, we get the first frame of the video sequence as the background frame. And then we'll always compare the current frame with the background one. But it will give us the result I've described above, which we obviously don't want very much. Our approach is to "move" the background frame to the current frame on the specified amount (I've used 1 level per frame). We move the background frame slightly in the direction of the current frame - we are changing colors of pixels in the background frame by one level per frame.
Collapse Copy Code// create filter
MoveTowards moveTowardsFilter = new MoveTowards( );
// move background towards current frame
moveTowardsFilter.OverlayImage = currentFrame;
Bitmap tmp = moveTowardsFilter.Apply( backgroundFrame );
// dispose old background
backgroundFrame.Dispose( );
backgroundFrame = tmp;And now, we can use the same approach we've used above. But, let me extend it slightly to get a more interesting result.
Collapse Copy Code// create processing filters sequence
FiltersSequence processingFilter = new FiltersSequence( );
processingFilter.Add( new Difference( backgroundFrame ) );
processingFilter.Add( new Threshold( 15 ) );
processingFilter.Add( new Opening( ) );
processingFilter.Add( new Edges( ) );
// apply the filter
Bitmap tmp1 = processingFilter.Apply( currentFrame );
// extract red channel from the original image
IFilter extrachChannel = new ExtractChannel( RGB.R );
Bitmap redChannel = extrachChannel.Apply( image );
// merge red channel with moving object borders
Merge mergeFilter = new Merge( );
mergeFilter.OverlayImage = tmp1;
Bitmap tmp2 = mergeFilter.Apply( redChannel );
// replace red channel in the original image
ReplaceChannel replaceChannel = new ReplaceChannel( RGB.R );
replaceChannel.ChannelImage = tmp2;
Bitmap tmp3 = replaceChannel.Apply( image );
Now it looks much better!
There is another approach based on the idea. As in the previous cases, we have an original frame and a gray scaled version of it and of the background frame. But let's apply Pixellate filter to the current frame and to the background before further processing.
Collapse Copy Code// create filter
IFilter pixellateFilter = new Pixellate( );
// apply the filter
Bitmap newImage = pixellateFilter( image );So, we have pixellated versions of the current and background frames. Now, we need to move the background frame towards the current frame as we were doing before. The next change is only the main processing step:
Collapse Copy Code// create processing filters sequence
FiltersSequence processingFilter = new FiltersSequence( );
processingFilter.Add( new Difference( backgroundFrame ) );
processingFilter.Add( new Threshold( 15 ) );
processingFilter.Add( new Dilatation( ) );
processingFilter.Add( new Edges( ) );
// apply the filter
Bitmap tmp1 = processingFilter.Apply( currentFrame );After merging tmp1 image with the red channel of the original image, we'll get the following image:
May be it looks not so perfect as the previous one, but the approach has a great possibility for performance optimization.
Looking at the previous picture, we can see, that objects are highlighted with a curve, which represents the moving object's boundary. But sometimes it's more likely to get a rectangle of the object. Not only this, what to do if we want not just highlight the objects, but get their count, position, width and height? Recently I was thinking: "Hmmm, it's possible, but not so trivial". Don't be afraid, it's easy. It can be done using the BlobCounter class from my imaging library, which was developed recently. Using BlobCounter we can get the number of objects, their position and the dimension on a binary image. So, let's try to apply it. We'll apply it to the binary image containing moving objects, the result of Threshold filter.
Collapse Copy CodeBlobCounter blobCounter = new BlobCounter( );
...
// get object rectangles
blobCounter.ProcessImage( thresholdedImage );
Rectangle[] rects = BlobCounter.GetObjectRectangles( );
// create graphics object from initial image
Graphics g = Graphics.FromImage( image );
// draw each rectangle
using ( Pen pen = new Pen( Color.Red, 1 ) )
{
foreach ( Rectangle rc in rects )
{
g.DrawRectangle( pen, rc );
if ( ( rc.Width > 15 ) && ( rc.Height > 15 ) )
{
// here we can higligh large objects with something else
}
}
}
g.Dispose( );Here is the result of this small piece of code. Looks pretty. Oh, I forgot. In my original implementation, there is some code instead of that comment for processing large objects. So, we can see a small numbers on the objects.
[14.06.2006] There was a lot of complains that the idea of MoveTowards filter, which is used for updating background image, is hard to understand. So, I was thinking a little bit about changing this filter to something else, which is clearer to understand. And the solution is to use Morph filer, which became available in 2.4 version of AForge.Imaging library. The new filter has two benefits:
It is much more simpler to understand;
The implementation of the filter is more efficient, so the filter produce better performance.
The idea of the filter is to preserve specified percentage of the source filter and to add missing percentage from overlay image. So, if the filter was applied to source image with percent value equal to 60%, then the result image will contain 60% of source image and 40% of overlay image. Applying the filter with percent values around 90% makes background image changing continuously to current frame.
Motion Alarm
It is pretty easy to add motion alarm feature to all these motion detection algorithms. Each algorithm calculates a binary image containing difference between current frame and the background one. So, the only we need is to just calculate the amount of white pixels on this difference image.
Collapse Copy Code// Calculate white pixels
private int CalculateWhitePixels( Bitmap image )
{
int count = 0;
// lock difference image
BitmapData data = image.LockBits( new Rectangle( 0, 0, width, height ),
ImageLockMode.ReadOnly, PixelFormat.Format8bppIndexed );
int offset = data.Stride - width;
unsafe
{
byte * ptr = (byte *) data.Scan0.ToPointer( );
for ( int y = 0; y < height; y++ )
{
for ( int x = 0; x < width; x++, ptr++ )
{
count += ( (*ptr) >> 7 );
}
ptr += offset;
}
}
// unlock image
image.UnlockBits( data );
return count;
}For some algorithms it could be done even simpler. For example, in blob counting approach we can accumulate not the white pixels count, but the area of each detected object. Then, if the computed amount of changes is greater than a predefined value, we can fire an alarm event.
Video Saving
There are many different ways to process motion alarm event: just draw a blinking rectangle around the video, or play sound to attract attention. But, of course, the most useful one is video saving on motion detection. In the demo application I was using the AVIWriter class, which uses Video for Windows interop to provide AVI files saving capabilities. Here is the small sample of using the class to write small AVI file, which draw diagonal line:
Collapse Copy CodeSaveFileDialog sfd = new SaveFileDialog( );
if ( sfd.ShowDialog( ) == DialogResult.OK )
{
AVIWriter writer = new AVIWriter( "wmv3" );
try
{
writer.Open( sfd.FileName, 320, 240 );
Bitmap bmp = new Bitmap( 320, 240, PixelFormat.Format24bppRgb );
for ( int i = 0; i < 100; i++ )
{
bmp.SetPixel( i, i, Color.FromArgb( i, 0, 255 - i ) );
writer.AddFrame( bmp );
}
bmp.Dispose( );
}
catch ( ApplicationException ex )
{
}
writer.Dispose( );
}Note: In this small sample and in the demo application I was using Windows Media Video 9 VCM codec.
AForge.NET framework
The Motion Detection application is based on the AForge.NET framework, which provides all the filters and image processing routines used in this application. To get more information about the framework, you may read dedicated article on Code Project or visit project's home page, where you can get all the latest information about it, participate in a discussion group or submit issues or requests for enhancements.
Applications for motion detection
Some people ask me one question from time to time, which is a little bit strange to me. The question is "What is the application for motion detectors". There is a lot to do with them and it depends on the imagination. One of the most straight forward applications is video surveillance, but it is not the only one. Since the first release of this application, I've received many e-mails from different people, who applied this application to incredible things. Some of them have their own articles, so you can take a look:
"Laser Gesture Recognition" by Ashish Derhgawen;
"Everyone Loves Babies! Webcams and Motion Detection" by Scott Hanselman;
"Motion-Detecting, Blood Squirting Halloween Skull" by Brian Peek.
Conclusion
I've described only ideas here. To use these ideas in real applications, you need to optimize its realization. I've used an image processing library for simplicity, it's not a video processing library. Besides, the library allows me to research different areas more quickly, than to write optimized solutions from the beginning. A small sample of optimization can be found in the sources.
History
[20.04.2007] - 1.5
Project converted to .NET 2.0;
Integrated with AForge.NET framework;
Motion detectors updated to use new features of AForge.NET to speed-up processing.
[15.06.2006] - 1.4 - Added fifth method based of Morph filter of AForge.Imaging library.
[08.04.2006] - 1.3 - Motion alarm and video saving.
[22.08.2005] - 1.2 - Added fourth method (getting objects' rectangles with blob counter).
[01.06.2005] - 1.1 - Added support of local capture devices and MMS streams.
[30.04.2005] - 1.0 - Initial release.
License
This article, along with any associated source code and files, is licensed under The GNU General Public License (GPLv3)
There are many approaches for motion detection in a continuous video stream. All of them are based on comparing of the current video frame with one from the previous frames or with something that we'll call background. In this article, I'll try to describe some of the most common approaches.
In description of these algorithms I'll use the AForge.NET framework, which is described in some other articles on Code Project: [1], [2]. So, if you are common with it, it will only help.
The demo application supports the following types of video sources:
AVI files (using Video for Windows, interop library is included);
updating JPEG from internet cameras;
MJPEG (motion JPEG) streams from different internet cameras;
local capture device (USB cameras or other capture devices, DirectShow interop library is included).
Algorithms
One of the most common approaches is to compare the current frame with the previous one. It's useful in video compression when you need to estimate changes and to write only the changes, not the whole frame. But it is not the best one for motion detection applications. So, let me describe the idea more closely.
Assume that we have an original 24 bpp RGB image called current frame (image), a grayscale copy of it (currentFrame) and previous video frame also gray scaled (backgroundFrame). First of all, let's find the regions where these two frames are differing a bit. For the purpose we can use Difference and Threshold filters.
Collapse Copy Code// create filters
Difference differenceFilter = new Difference( );
IFilter thresholdFilter = new Threshold( 15 );
// set backgroud frame as an overlay for difference filter
differenceFilter.OverlayImage = backgroundFrame;
// apply the filters
Bitmap tmp1 = differenceFilter.Apply( currentFrame );
Bitmap tmp2 = thresholdFilter.Apply( tmp1 );On this step we'll get an image with white pixels on the place where the current frame is different from the previous frame on the specified threshold value. It's already possible to count the pixels, and if the amount of it will be greater than a predefined alarm level we can signal about a motion event.
But most cameras produce a noisy image, so we'll get motion in such places, where there is no motion at all. To remove random noisy pixels, we can use an Erosion filter, for example. So, we'll get now mostly only the regions where the actual motion was.
Collapse Copy Code// create filter
IFilter erosionFilter = new Erosion( );
// apply the filter
Bitmap tmp3 = erosionFilter.Apply( tmp2 );The simplest motion detector is ready! We can highlight the motion regions if needed.
Collapse Copy Code// extract red channel from the original image
IFilter extrachChannel = new ExtractChannel( RGB.R );
Bitmap redChannel = extrachChannel.Apply( image );
// merge red channel with motion regions
Merge mergeFilter = new Merge( );
mergeFilter.OverlayImage = tmp3;
Bitmap tmp4 = mergeFilter.Apply( redChannel );
// replace red channel in the original image
ReplaceChannel replaceChannel = new ReplaceChannel( RGB.R );
replaceChannel.ChannelImage = tmp4;
Bitmap tmp5 = replaceChannel.Apply( image );Here is the result of it:
From the above picture we can see the disadvantages of the approach. If the object is moving smoothly we'll receive small changes from frame to frame. So, it's impossible to get the whole moving object. Things become worse, when the object is moving so slowly, when the algorithms will not give any result at all.
There is another approach. It's possible to compare the current frame not with the previous one but with the first frame in the video sequence. So, if there were no objects in the initial frame, comparison of the current frame with the first one will give us the whole moving object independently of its motion speed. But, the approach has a big disadvantage - what will happen, if there was, for example, a car on the first frame, but then it is gone? Yes, we'll always have motion detected on the place, where the car was. Of course, we can renew the initial frame sometimes, but still it will not give us good results in the cases where we can not guarantee that the first frame will contain only static background. But, there can be an inverse situation. If I'll put a picture on the wall in the room? I'll get motion detected until the initial frame will be renewed.
The most efficient algorithms are based on building the so called background of the scene and comparing each current frame with the background. There are many approaches to build the scene, but most of them are too complex. I'll describe here my approach for building the background. It's rather simple and can be realized very quickly.
As in the previous case, let's assume that we have an original 24 bpp RGB image called current frame (image), a grayscale copy of it (currentFrame) and a background frame also gray scaled (backgroundFrame). At the beginning, we get the first frame of the video sequence as the background frame. And then we'll always compare the current frame with the background one. But it will give us the result I've described above, which we obviously don't want very much. Our approach is to "move" the background frame to the current frame on the specified amount (I've used 1 level per frame). We move the background frame slightly in the direction of the current frame - we are changing colors of pixels in the background frame by one level per frame.
Collapse Copy Code// create filter
MoveTowards moveTowardsFilter = new MoveTowards( );
// move background towards current frame
moveTowardsFilter.OverlayImage = currentFrame;
Bitmap tmp = moveTowardsFilter.Apply( backgroundFrame );
// dispose old background
backgroundFrame.Dispose( );
backgroundFrame = tmp;And now, we can use the same approach we've used above. But, let me extend it slightly to get a more interesting result.
Collapse Copy Code// create processing filters sequence
FiltersSequence processingFilter = new FiltersSequence( );
processingFilter.Add( new Difference( backgroundFrame ) );
processingFilter.Add( new Threshold( 15 ) );
processingFilter.Add( new Opening( ) );
processingFilter.Add( new Edges( ) );
// apply the filter
Bitmap tmp1 = processingFilter.Apply( currentFrame );
// extract red channel from the original image
IFilter extrachChannel = new ExtractChannel( RGB.R );
Bitmap redChannel = extrachChannel.Apply( image );
// merge red channel with moving object borders
Merge mergeFilter = new Merge( );
mergeFilter.OverlayImage = tmp1;
Bitmap tmp2 = mergeFilter.Apply( redChannel );
// replace red channel in the original image
ReplaceChannel replaceChannel = new ReplaceChannel( RGB.R );
replaceChannel.ChannelImage = tmp2;
Bitmap tmp3 = replaceChannel.Apply( image );
Now it looks much better!
There is another approach based on the idea. As in the previous cases, we have an original frame and a gray scaled version of it and of the background frame. But let's apply Pixellate filter to the current frame and to the background before further processing.
Collapse Copy Code// create filter
IFilter pixellateFilter = new Pixellate( );
// apply the filter
Bitmap newImage = pixellateFilter( image );So, we have pixellated versions of the current and background frames. Now, we need to move the background frame towards the current frame as we were doing before. The next change is only the main processing step:
Collapse Copy Code// create processing filters sequence
FiltersSequence processingFilter = new FiltersSequence( );
processingFilter.Add( new Difference( backgroundFrame ) );
processingFilter.Add( new Threshold( 15 ) );
processingFilter.Add( new Dilatation( ) );
processingFilter.Add( new Edges( ) );
// apply the filter
Bitmap tmp1 = processingFilter.Apply( currentFrame );After merging tmp1 image with the red channel of the original image, we'll get the following image:
May be it looks not so perfect as the previous one, but the approach has a great possibility for performance optimization.
Looking at the previous picture, we can see, that objects are highlighted with a curve, which represents the moving object's boundary. But sometimes it's more likely to get a rectangle of the object. Not only this, what to do if we want not just highlight the objects, but get their count, position, width and height? Recently I was thinking: "Hmmm, it's possible, but not so trivial". Don't be afraid, it's easy. It can be done using the BlobCounter class from my imaging library, which was developed recently. Using BlobCounter we can get the number of objects, their position and the dimension on a binary image. So, let's try to apply it. We'll apply it to the binary image containing moving objects, the result of Threshold filter.
Collapse Copy CodeBlobCounter blobCounter = new BlobCounter( );
...
// get object rectangles
blobCounter.ProcessImage( thresholdedImage );
Rectangle[] rects = BlobCounter.GetObjectRectangles( );
// create graphics object from initial image
Graphics g = Graphics.FromImage( image );
// draw each rectangle
using ( Pen pen = new Pen( Color.Red, 1 ) )
{
foreach ( Rectangle rc in rects )
{
g.DrawRectangle( pen, rc );
if ( ( rc.Width > 15 ) && ( rc.Height > 15 ) )
{
// here we can higligh large objects with something else
}
}
}
g.Dispose( );Here is the result of this small piece of code. Looks pretty. Oh, I forgot. In my original implementation, there is some code instead of that comment for processing large objects. So, we can see a small numbers on the objects.
[14.06.2006] There was a lot of complains that the idea of MoveTowards filter, which is used for updating background image, is hard to understand. So, I was thinking a little bit about changing this filter to something else, which is clearer to understand. And the solution is to use Morph filer, which became available in 2.4 version of AForge.Imaging library. The new filter has two benefits:
It is much more simpler to understand;
The implementation of the filter is more efficient, so the filter produce better performance.
The idea of the filter is to preserve specified percentage of the source filter and to add missing percentage from overlay image. So, if the filter was applied to source image with percent value equal to 60%, then the result image will contain 60% of source image and 40% of overlay image. Applying the filter with percent values around 90% makes background image changing continuously to current frame.
Motion Alarm
It is pretty easy to add motion alarm feature to all these motion detection algorithms. Each algorithm calculates a binary image containing difference between current frame and the background one. So, the only we need is to just calculate the amount of white pixels on this difference image.
Collapse Copy Code// Calculate white pixels
private int CalculateWhitePixels( Bitmap image )
{
int count = 0;
// lock difference image
BitmapData data = image.LockBits( new Rectangle( 0, 0, width, height ),
ImageLockMode.ReadOnly, PixelFormat.Format8bppIndexed );
int offset = data.Stride - width;
unsafe
{
byte * ptr = (byte *) data.Scan0.ToPointer( );
for ( int y = 0; y < height; y++ )
{
for ( int x = 0; x < width; x++, ptr++ )
{
count += ( (*ptr) >> 7 );
}
ptr += offset;
}
}
// unlock image
image.UnlockBits( data );
return count;
}For some algorithms it could be done even simpler. For example, in blob counting approach we can accumulate not the white pixels count, but the area of each detected object. Then, if the computed amount of changes is greater than a predefined value, we can fire an alarm event.
Video Saving
There are many different ways to process motion alarm event: just draw a blinking rectangle around the video, or play sound to attract attention. But, of course, the most useful one is video saving on motion detection. In the demo application I was using the AVIWriter class, which uses Video for Windows interop to provide AVI files saving capabilities. Here is the small sample of using the class to write small AVI file, which draw diagonal line:
Collapse Copy CodeSaveFileDialog sfd = new SaveFileDialog( );
if ( sfd.ShowDialog( ) == DialogResult.OK )
{
AVIWriter writer = new AVIWriter( "wmv3" );
try
{
writer.Open( sfd.FileName, 320, 240 );
Bitmap bmp = new Bitmap( 320, 240, PixelFormat.Format24bppRgb );
for ( int i = 0; i < 100; i++ )
{
bmp.SetPixel( i, i, Color.FromArgb( i, 0, 255 - i ) );
writer.AddFrame( bmp );
}
bmp.Dispose( );
}
catch ( ApplicationException ex )
{
}
writer.Dispose( );
}Note: In this small sample and in the demo application I was using Windows Media Video 9 VCM codec.
AForge.NET framework
The Motion Detection application is based on the AForge.NET framework, which provides all the filters and image processing routines used in this application. To get more information about the framework, you may read dedicated article on Code Project or visit project's home page, where you can get all the latest information about it, participate in a discussion group or submit issues or requests for enhancements.
Applications for motion detection
Some people ask me one question from time to time, which is a little bit strange to me. The question is "What is the application for motion detectors". There is a lot to do with them and it depends on the imagination. One of the most straight forward applications is video surveillance, but it is not the only one. Since the first release of this application, I've received many e-mails from different people, who applied this application to incredible things. Some of them have their own articles, so you can take a look:
"Laser Gesture Recognition" by Ashish Derhgawen;
"Everyone Loves Babies! Webcams and Motion Detection" by Scott Hanselman;
"Motion-Detecting, Blood Squirting Halloween Skull" by Brian Peek.
Conclusion
I've described only ideas here. To use these ideas in real applications, you need to optimize its realization. I've used an image processing library for simplicity, it's not a video processing library. Besides, the library allows me to research different areas more quickly, than to write optimized solutions from the beginning. A small sample of optimization can be found in the sources.
History
[20.04.2007] - 1.5
Project converted to .NET 2.0;
Integrated with AForge.NET framework;
Motion detectors updated to use new features of AForge.NET to speed-up processing.
[15.06.2006] - 1.4 - Added fifth method based of Morph filter of AForge.Imaging library.
[08.04.2006] - 1.3 - Motion alarm and video saving.
[22.08.2005] - 1.2 - Added fourth method (getting objects' rectangles with blob counter).
[01.06.2005] - 1.1 - Added support of local capture devices and MMS streams.
[30.04.2005] - 1.0 - Initial release.
License
This article, along with any associated source code and files, is licensed under The GNU General Public License (GPLv3)
Subscribe to:
Posts (Atom)