Hi Guys,
I posted a video before showing how you can use a robot to draw a picture.
The whole purpose that video was just to showcase the power of robotics and how UiPath robot can control the mouse even at pixel level.
The response to that video was really great and lot of people had requested me to explain the logic behind the painter robot.In this post, I wish to show step by step analysis of the drawing robot workflow.
Below video explains the logic in detail:
https://www.linkedin.com/feed/update/urn:li:activity:6384009728531496960/
His post reminded me what I had done 10 years back when I was exploring VBA in Excel. At that time, I did a fun project to map a picture in Excel by coloring the Excel cells using VBA.
In my Excel image mapping project, I used VBA code to read the XY cordinates and RGB color combination of each pixel of a picture into an array, then I colored each cell in a worksheet using the array as if it is a pixel of a picture. Then if you reduce the zooming of the worksheet to 10%, it will exactly looks like the same picture.
This made me thinking that why can't I upgrade my old project to a robot, so I can showcase the power of robots while I'm presenting to my client or in my training sessions.
In fact, UiPath also uses the Microsoft .NET background, So it should be an easy task to upgrade my VBA project to UiPath.
The I assigned 2 hours to do this task.
But when it comes to reality, there were few constraints:
> I was using Excel, here I need to use Microsoft Paint
> I was reading the actual colors of the picture in my old project and making the same color in Excel cell. Since I'm using MS Paint to draw the picture, there is NO easy shortcut available to change the color of the brush everytime when the color changes. If I use the UI elements in Paint to change the color on each color, my robot will take a day to paint a picture. So I decided to read only the cordinates of the darker shades of the picture and map only those dots in Paint to imitate the picture.
The outcome is not so fantastic like my old project, but still it's enough to showcase the power of the UiPath robot.
Before starting, you should know how a computer saves a picture. In a common uncompressed bitmap, the image is stored as a series of dots which is also known as pixels. Each pixel is a very tiny dot or a square and a color is assigned to each pixel. Then the pixels are arranged in a pattern to form an image. You can see the pixels easily by opening a picture in Paintbrush and zoom In. So each pixel has some properties like it's position on the image, it is saved as XY scatter points. Then each pixel will have it's color. In easy way, you can see each color is a combination of Red, Green and Blue which varies from zero to 255.
In short, if you know the XY cordinates of each pixel in an image and also the color of each pixel, you can easily recreate an image.
In our project, you have to write a code to get the cordinates only since we are not going to use the color. Then write a code to mark each cordinate in Paintbrush.
Enough of gyan. Now let us look into the UiPath workflow.
You can download our project workflow from this link http://zipansion.com/hA5S
In first step, I inserted an activity to ask the user to select the picture he wished to sketch. Currently, I put a filter for only J P G files. You can change the filter as you wish. But make sure that the image you select is not a compressed bitmap.
Then the file path is saved to a string variable.
Our next step is to read the cordinates of the selected image. I create a small module for doing this task. I pass the image file path into this module and the module will return a Datatable with the cordinates to be marked.
Now let us look how this module works .
In first, I created a datatable object and created two columns for saving X and Y cordinate values.
Then we create a bitmap object using the image file we selected.
Then we need to find the height and width of the image and saves into variables.
Next, create nested loops to loop through each pixels of the image.
Within the loop, on each pixel, we use Get Pixel method to find the color of that pixel.
The color is assigned to a color variable.
The color variable has the properties to find its RED, Green and Blue of the color.
As we told earlier, RGB varies from zero to 255, we are finding only the darker shades of the picture. So we check if the pixel's Red, Green and Blue are less than 20, which are more darker. Then save the cordinates to a new row in the datatable.
If you want to get more pixel quality to your picture, you can increase the color range from 20, but the robot will take more time to finish the drawing since it will increase the number of pixels to be marked.
When robot finishes the loops, you have a datatable with all the cordinates to be marked.
Now let us go back to the main workflow.
In next step, I used Open application activity to open Paintbrush.
the next step is to change the color of the brush, if you want the robot to sketch the entire picture in a specific color.
In the next step, we have a loop to go through each row in our datatable.
Inside the loop, we use click activity to mark each pixel in paintbrush by changing X and Y values in the cursor position properties.
When the loop finishes, your robot will finish marking of all the pixels of your image.
You can download the workflow from the link below if you wish to refer. But I would suggest you to understand the logic and try it yourself.
Thanks for reading my post how you can use a UiPath robot to draw a picture.
I posted a video before showing how you can use a robot to draw a picture.
The whole purpose that video was just to showcase the power of robotics and how UiPath robot can control the mouse even at pixel level.
The response to that video was really great and lot of people had requested me to explain the logic behind the painter robot.In this post, I wish to show step by step analysis of the drawing robot workflow.
Below video explains the logic in detail:
Let's see how this idea came to my mind.
Even though I'm not an active social media person, I use LinkedIn couple of hours a week to get updated on the technology news. Couple of months back, A LinkedIn post made me surprised. It was the post of Alekh Barli, CEO of INTELLIBOT.IO where he showcased the capability of Intellibot Robot to control the mouse at pixel level by drawing a picture in MS Paint.https://www.linkedin.com/feed/update/urn:li:activity:6384009728531496960/
His post reminded me what I had done 10 years back when I was exploring VBA in Excel. At that time, I did a fun project to map a picture in Excel by coloring the Excel cells using VBA.
In my Excel image mapping project, I used VBA code to read the XY cordinates and RGB color combination of each pixel of a picture into an array, then I colored each cell in a worksheet using the array as if it is a pixel of a picture. Then if you reduce the zooming of the worksheet to 10%, it will exactly looks like the same picture.
This made me thinking that why can't I upgrade my old project to a robot, so I can showcase the power of robots while I'm presenting to my client or in my training sessions.
In fact, UiPath also uses the Microsoft .NET background, So it should be an easy task to upgrade my VBA project to UiPath.
The I assigned 2 hours to do this task.
But when it comes to reality, there were few constraints:
> I was using Excel, here I need to use Microsoft Paint
> I was reading the actual colors of the picture in my old project and making the same color in Excel cell. Since I'm using MS Paint to draw the picture, there is NO easy shortcut available to change the color of the brush everytime when the color changes. If I use the UI elements in Paint to change the color on each color, my robot will take a day to paint a picture. So I decided to read only the cordinates of the darker shades of the picture and map only those dots in Paint to imitate the picture.
The outcome is not so fantastic like my old project, but still it's enough to showcase the power of the UiPath robot.
Now let us go to the actual logic of the robot workflow.
Before starting, you should know how a computer saves a picture. In a common uncompressed bitmap, the image is stored as a series of dots which is also known as pixels. Each pixel is a very tiny dot or a square and a color is assigned to each pixel. Then the pixels are arranged in a pattern to form an image. You can see the pixels easily by opening a picture in Paintbrush and zoom In. So each pixel has some properties like it's position on the image, it is saved as XY scatter points. Then each pixel will have it's color. In easy way, you can see each color is a combination of Red, Green and Blue which varies from zero to 255.
In short, if you know the XY cordinates of each pixel in an image and also the color of each pixel, you can easily recreate an image.
In our project, you have to write a code to get the cordinates only since we are not going to use the color. Then write a code to mark each cordinate in Paintbrush.
Enough of gyan. Now let us look into the UiPath workflow.
You can download our project workflow from this link http://zipansion.com/hA5S
In first step, I inserted an activity to ask the user to select the picture he wished to sketch. Currently, I put a filter for only J P G files. You can change the filter as you wish. But make sure that the image you select is not a compressed bitmap.
Then the file path is saved to a string variable.
Our next step is to read the cordinates of the selected image. I create a small module for doing this task. I pass the image file path into this module and the module will return a Datatable with the cordinates to be marked.
Now let us look how this module works .
In first, I created a datatable object and created two columns for saving X and Y cordinate values.
Then we create a bitmap object using the image file we selected.
Then we need to find the height and width of the image and saves into variables.
Next, create nested loops to loop through each pixels of the image.
Within the loop, on each pixel, we use Get Pixel method to find the color of that pixel.
The color is assigned to a color variable.
The color variable has the properties to find its RED, Green and Blue of the color.
As we told earlier, RGB varies from zero to 255, we are finding only the darker shades of the picture. So we check if the pixel's Red, Green and Blue are less than 20, which are more darker. Then save the cordinates to a new row in the datatable.
If you want to get more pixel quality to your picture, you can increase the color range from 20, but the robot will take more time to finish the drawing since it will increase the number of pixels to be marked.
When robot finishes the loops, you have a datatable with all the cordinates to be marked.
Now let us go back to the main workflow.
In next step, I used Open application activity to open Paintbrush.
the next step is to change the color of the brush, if you want the robot to sketch the entire picture in a specific color.
In the next step, we have a loop to go through each row in our datatable.
Inside the loop, we use click activity to mark each pixel in paintbrush by changing X and Y values in the cursor position properties.
When the loop finishes, your robot will finish marking of all the pixels of your image.
You can download the workflow from the link below if you wish to refer. But I would suggest you to understand the logic and try it yourself.
Thanks for reading my post how you can use a UiPath robot to draw a picture.
No comments:
Post a Comment