设置
快速交货
免运费
国际贸易术语
支付类型
市场产品
制造商零件编号 SEN-15325
OPENMV H7 CAMERA
SparkFun Electronics
Getting Started with the OpenMV Camera: Machine Vision with MicroPython
2020-09-08 | By ShawnHymel
License: Attribution
Machine vision is a collection of tools and techniques used to automatically analyze and interpret images to help computers make decisions about its surroundings. You might find machine vision used in things like automated inspection, robot guidance, and process control.
OpenMV is a self-contained machine vision platform that includes a camera and a powerful microcontroller. Out of the box, it runs MicroPython and includes an application programming interface (API) with various machine vision functions that make interpreting images much easier. The rest of this tutorial will walk you through the process of setting up the OpenMV camera, running MicroPython code, and using machine learning to identify person-like figures in captured images.
You can also see this tutorial in video form here:
Required Hardware
For this project, you will need the following hardware:
IDE Installation
OpenMV comes with its own integrated development environment (IDE). To install it, head to https://openmv.io/pages/download. Download and run the installer for your operating system, accepting all the defaults.
Hello World
Use the USB cable to connect your OpenMV camera module to your computer.
Open the OpenMV IDE, and you should see that it starts with a pre-made “Hello, World!” application in the main editor window. In the bottom-left corner, you should see an icon that looks like 2 plugs connected together. This symbol indicates that the OpenMV camera is not connected.
If the camera is not plugged in and you click on that double-plug symbol, the IDE will ask you if you need help to un-brick your camera module.
If the camera is plugged in, you should see a USB symbol over the double-plug icon. Click on the double-plug icon. For a new camera module, the IDE will likely ask you if you wish to upgrade the camera’s firmware. You should click OK and accept the defaults to upgrade the OpenMV firmware. Follow all of the instructions, including waiting at the end for the OpenMV’s LED to flash blue.
When the firmware upgrade process is done, the double-plug icon should look like a single plug icon, which indicates that the IDE has connected to the OpenMV camera module. You can then click the Play button in the bottom-left corner to run the code in the IDE on the OpenMV camera.
Let’s take a look at the provided “Hello, World!” code:
Copy Code
# Hello World Example
#
# Welcome to the OpenMV IDE! Click on the green run arrow button below to run the script!
import sensor, image, time
sensor.reset() # Reset and initialize the sensor.
sensor.set_pixformat(sensor.RGB565) # Set pixel format to RGB565 (or GRAYSCALE)
sensor.set_framesize(sensor.QVGA) # Set frame size to QVGA (320x240)
sensor.skip_frames(time = 2000) # Wait for settings take effect.
clock = time.clock() # Create a clock object to track the FPS.
while(True):
clock.tick() # Update the FPS clock.
img = sensor.snapshot() # Take a picture and return the image.
print(clock.fps()) # Note: OpenMV Cam runs about half as fast when connected
# to the IDE. The FPS should increase once disconnected.
We can see that it imports the sensor, image, and time modules from the OpenMV MicroPython library. From there, it initializes the camera sensor by resetting it and setting the color depth and resolution. The skip_frames(time = 2000) function tells the module to wait for a few seconds to allow the sensor to settle. After that, we create a clock object, which allows us to measure the frames per second (FPS) of the camera and associated code.
In the main while loop, we first update the FPS clock and then capture an image from the camera. This image is saved in the dummy variable img. By doing this, we can update the module’s frame buffer, which is shown in the upper-right corner (don’t forget to remove the OpenMV's lens cap!). Finally, we print the FPS value to the serial terminal.
See the OpenMV documentation page to learn more about the different functions within the API.
If you click the Serial Terminal button at the bottom of the IDE, you should get a console pane pop up and display the current FPS of the camera module.
Adjust Lens
If you notice that the image in the frame buffer pane looks blurry, dirty, or out of focus, you can adjust the lens on the camera module. Use a small Phillips screwdriver to loosen the set screw on the side of the lens. Then, you can slowly rotate the lens to adjust the focus.
Note that the camera module can be completely removed from the OpenMV board. This allows you to replace it with another camera module, such as a FLIR Lepton infrared camera.
Running MicroPython
Out of the box, the OpenMV module runs MicroPython. To support MicroPython deployment, the OpenMV enumerates as a USB mass storage device on your computer. If you look at the drive’s root directory, you should see a few files, including main.py, which is the MicroPython program that has been deployed to the module.
Please note that the enumerated drive has very limited space (around 100 kB). As a result, it is recommended that you do not use this drive to store anything other than your MicroPython code. If you try to store large files (such as images taken from the camera) on this drive, you might corrupt the filesystem.
If you would like to add more space, you can add a microSD to the OpenMV camera module. Note that when you do this, the microSD replaces the default, internal filesystem, which means you can store main.py on the SD card.
Person Detection
Identifying people in a frame has a variety of uses, from security cameras to automatic lighting systems to experiential marketing. Many security cameras require an extra cost to enable this feature. However, we can create our own person detection system using OpenMV almost straight out of the box!
In the OpenMV IDE, go to File > Examples > Machine Learning > tf_person_detection_search_whole_window. Feel free to look through this example to see how OpenMV is using the TensorFlow Lite module to perform inference, which looks at each frame for a person-like figure.
We will create a new project based on this example. Replace the code with the following:
Copy Code
# TensorFlow Lite Person Dection Example
#
# Google's Person Detection Model detects if a person is in view.
#
# In this example we slide the detector window over the image and get a list
# of activations. Note that use a CNN with a sliding window is extremely compute
# expensive so for an exhaustive search do not expect the CNN to be real-time.
import pyb, sensor, image, time, os, tf
# Settings
person_threshold = 0.7
# LED setting (1: red, 2: green, 3: blue, 4: IR)
led = pyb.LED(1)
led.off()
sensor.reset() # Reset and initialize the sensor.
sensor.set_pixformat(sensor.GRAYSCALE) # Set pixel format to RGB565 (or GRAYSCALE)
sensor.set_framesize(sensor.QVGA) # Set frame size to QVGA (320x240)
sensor.set_windowing((240, 240)) # Set 240x240 window.
sensor.skip_frames(time=2000) # Let the camera adjust.
# Load the built-in person detection network (the network is in your OpenMV Cam's firmware).
net = tf.load('person_detection')
labels = ['unsure', 'person', 'no_person']
# Star the clock to measure FPS
clock = time.clock()
# Main while loop
while(True):
# Measure time
clock.tick()
# Get image from camera
img = sensor.snapshot()
# net.classify() will run the network on an roi in the image (or on the whole image if the roi is not
# specified). A classification score output vector will be generated for each location. At each scale the
# detection window is moved around in the ROI using x_overlap (0-1) and y_overlap (0-1) as a guide.
# If you set the overlap to 0.5 then each detection window will overlap the previous one by 50%. Note
# the computational work load goes WAY up the more overlap. Finally, for multi-scale matching after
# sliding the network around in the x/y dimensions the detection window will shrink by scale_mul (0-1)
# down to min_scale (0-1). For example, if scale_mul is 0.5 the detection window will shrink by 50%.
# Note that at a lower scale there's even more area to search if x_overlap and y_overlap are small...
# default settings just do one detection... change them to search the image...
for obj in net.classify(img, min_scale=1.0, scale_mul=0.0, x_overlap=0.0, y_overlap=0.0):
# Give classification scores
print("**********\nDetections at [x=%d,y=%d,w=%d,h=%d]" % obj.rect())
for i in range(len(obj.output())):
print("%s = %f" % (labels[i], obj.output()[i]))
# Highlight identified object
img.draw_rectangle(obj.rect())
img.draw_string(obj.x()+3, obj.y()-1, labels[obj.output().index(max(obj.output()))], mono_space = False)
# Light LED if person detected
idx = labels.index('person')
if obj.output()[idx] > person_threshold:
led.on()
else:
led.off()
print(clock.fps(), "fps")
In this example, we create an LED object, which allows us to control the onboard tri-color (RGB) LED. Like before, we initialize the camera module, but we set it to grayscale and crop the frame to 240x240 pixels.
Next, we load the person_detection model, which comes with the OpenMV MicroPython library. The labels associated with the output of this model are “unsure,” “person,” and “no_person.” We use the classify() function to perform inference on each frame. The model will output 3 probabilities, corresponding to the 3 labels. If our “person” probability is over our threshold (set at 0.7), we will turn on the LED. Otherwise, we turn off the LED.
Click the Run button in the IDE, and you should see the frame buffer pane show you what the camera sees. Click the Serial Terminal button to see the output of the model, which includes the probability measure for each label as well as the FPS.
Upload Program to OpenMV
When you press the Play button in the IDE, it will run the code on the OpenMV board, but it will not upload the program to run independently of the IDE. If you would like to run your program on the OpenMV without it being connected to the IDE, click Tools > Save open script to OpenMV Cam (as main.py). This will overwrite main.py on the camera’s filesystem with whatever code you have open in the IDE.
If you are running Windows, it is recommended that you right-click on the drive in File Explorer and select Eject. This action will force Windows to finish writing and syncing the drive so that you may remove the OpenMV without corrupting the filesystem.
At this point, you can plug the camera into any 5V USB power source to run the program. When you (or another person) walk into the field of view of the camera, you should see the red LED light up. The LED should turn off when there are no people in the field of view.
Resources and Going Further
Hopefully, this guide has helped you get started using the OpenMV camera as well as act as an introduction to running TensorFlow Lite to perform basic machine learning functions. Here are a few links to help you in your machine vision journey:
Have questions or comments? Continue the conversation on TechForum, DigiKey's online community and technical resource.
Visit TechForum
