i’m doing open CV in python in blender with this code:
# Python code to read image
import cv2
import bpy
# Set the output file path of your rendered viewport image
output_file = "C:/path/to/output.png"
# Set the viewport resolution
bpy.context.scene.render.resolution_x = 1920
bpy.context.scene.render.resolution_y = 1080
# Set the output format
bpy.context.scene.render.image_settings.file_format = "PNG"
# Render the viewport and save the result
bpy.ops.render.opengl(write_still=True)
bpy.data.images["Render Result"].save_render(output_file)
# Loads an image into OpenCV environment
img = cv2.imread(output_file, cv2.IMREAD_COLOR)
# Resizes the image
dim = (1280, 720)
resized = cv2.resize(img, dim, interpolation = cv2.INTER_AREA)
print('Resized Dimensions : ',resized.shape)
# Shows the image in an OpenCV windoww
cv2.imshow("Resized image", resized)
# Necessary to close window with red X on top right
if cv2.waitKey(10) & 0xFF == ord('q'):
cap.release()
cv2.destroyAllWindows()
in the code I do a render of the viewport and then save it to my hard drive and then open it up in open CV code with the imread function…here is What it looks like when I do…see all the spheres are in the same position in my screenshot of the viewport, above, as they are in the image below
and I plan to do an openCV function to find the red sphere by color and use the openCV “rectangle” function to draw a rectangle around it in the image… as I will have the coordinates in the image from the find color function to draw the rectangle…
since it is a image of the viewport as it is exactly when I rendered this image I wondered if there’s any way once I have the coordinates in the rendered image that I got with openCV by doing the color detection and drawing a rectangle around the red sphere to translate that into finding the exact coordinates of the sphere in the blender viewport so essentially I could track any red sphere in a scene And then using blender python select find the sphere in the viewport and select it…
if you need more information let me know And any help on this is appreciated
I think you’ll run into a problem, since you find the position of the sphere in 2D / screenspace, you need to convert that back to 3D. And how could you account for the depth ( distance to camera ?)
probably if you render a position pass you can check it’s pixels to get the position of the sphere.
That seems the simplest solution…
can you tell me how I go about rendering a position pass and give me and give me as many step by step instructions as possible to do that to do that?
well I’m adding opencv functionality to my add on so I could use it to find and activate diffuse image texture nodes on all objects, materials, so you see color image textures in the viewport regardless of group node setup
and I’m trying to figure out some cool stuff I could do with it… so I can start adding a lot of cool opencv functionality and save my customers a lot of button clicks… I programmed an open CV library, a common lisp wrapper for C++ , So I know about open CV but as far as using in conjunction with the 3D viewport and blender that’s kind of a different thing…
A slightly different way of doing it might be to use Cryptomattes. With them, you can render an image with a mask for each object. So you could find every pixel that belongs to the red sphere and draw a rectangle around those.
I am not sure if you can do that in the viewport directly. It might be necessary to use a script to place a camera in the position where the viewport camera is (with the same settings) and render like that.
Thank you so much I’m gonna search on YouTube for tutorials on these things if you have any if you could throw them here id appreciate that but thanks for getting me started… The open CV part of my addon…is in Beta right now but it’s a really simple solid way to install opencv in blender whenever you want if you PM me I’ll give you a free code or send you a copy I don’t know if the gum road free codes are working good… It’ll keep getting better I’m a Tech lover and I know how to find the best programmers to help me if I can’t do it myself
I’m an efficiency expert and I’m building in all my tricks blender that save button clicks as far as fast production of animation scenes… So you could produce a lot of content and grow your YouTube channel better and approach your VFX profession better…It’s kind of all the knowledge that I’m gaining because I’m creating my own sitcom in blender and all the stuff I’m learning about how I could save button clicks from doing it… I’m creating the making a sitcom in blender series on youtube… part 1 & 2 has a bunch of good ideas as far as saving button clicks …the more button clicks you’ve save the more fun you have!
