使用卷積神經(jīng)網(wǎng)絡預防疲勞駕駛事故
[導讀]作者|小白來源|小白學視覺疲勞駕駛：一個嚴重的問題美國國家公路交通安全管理局估計，每年有91,000起車禍涉及疲勞駕駛的司機，造成約50,000人受傷和近800人死亡。此外，每24名成年司機中就有1人報告在過去30天內(nèi)在駕駛時睡著了。研究甚至發(fā)現(xiàn)，超過20個小時不睡覺相當于血液酒...
疲勞駕駛：一個嚴重的問題
美國國家公路交通安全管理局估計，每年有 91,000 起車禍涉及疲勞駕駛的司機，造成約50,000 人受傷和近 800 人死亡。此外，每 24 名成年司機中就有 1 人報告在過去 30 天內(nèi)在駕駛時睡著了。研究甚至發(fā)現(xiàn)，超過20個小時不睡覺相當于血液酒精濃度為0.08％——美國的法律規(guī)定的上限。由于這個嚴重的問題，我和一組其他數(shù)據(jù)科學家開始開發(fā)一種神經(jīng)網(wǎng)絡，可以檢測眼睛是否閉著，當與計算機視覺結(jié)合使用時，可以檢測活人是否閉著眼睛超過一秒鐘。這種技術對于任何對提高駕駛安全性感興趣的人都很有用，包括商業(yè)和日常司機、汽車公司和汽車保險公司。目錄構建卷積神經(jīng)網(wǎng)絡網(wǎng)絡攝像頭應用程序數(shù)據(jù)采集我們使用了多個來源的完整面部數(shù)據(jù)，即麻省大學阿默斯特分校的睜眼面部數(shù)據(jù)和南京大學的閉眼面部數(shù)據(jù)。然后，我們使用一個簡單的 Python 函數(shù)從這個數(shù)據(jù)集中裁剪出眼睛，只剩下 30,000 多張裁剪后的眼睛圖像。我們?yōu)槊總€圖像裁剪添加了一個緩沖區(qū)，不僅可以獲取眼睛，還可以獲取眼睛周圍的區(qū)域。此裁剪功能稍后將重新用于網(wǎng)絡攝像頭部分。# For loop going through each image file for folder in os.listdir(folders): for file in os.listdir(folders '/' folder):
# Using Facial Recognition Library on Image image = face_recognition.load_image_file(folders '/' folder '/' file) # create a variable for the facial feature coordinates face_landmarks_list = face_recognition.face_landmarks(image)
# create a placeholder list for the eye coordinates eyes = [] try: eyes.append(face_landmarks_list[0]['left_eye']) eyes.append(face_landmarks_list[0]['right_eye']) except: continue # establish the max x and y coordinates of the eye for eye in eyes: x_max = max([coordinate[0] for coordinate in eye]) x_min = min([coordinate[0] for coordinate in eye]) y_max = max([coordinate[1] for coordinate in eye]) y_min = min([coordinate[1] for coordinate in eye]) # establish the range of x and y coordinates x_range = x_max - x_min y_range = y_max - y_min
# to make sure the full eye is captured, # calculate the coordinates of a square that has 50% # cushion added to the axis with a larger range if x_range > y_range: right = round(.5*x_range) x_max left = x_min - round(.5*x_range) bottom = round(((right-left) - y_range))/2 y_max top = y_min - round(((right-left) - y_range))/2 else: bottom = round(.5*y_range) y_max top = y_min - round(.5*y_range) right = round(((bottom-top) - x_range))/2 x_max left = x_min - round(((bottom-top) - x_range))/2
#crop original image using the cushioned coordinates im = Image.open(folders '/' folder '/' file) im = im.crop((left, top, right, bottom))
# resize image for input into our model im = im.resize((80,80))
# save file to output folder im.save('yourfolderforcroppedeyes')
# increase count for iterative file saving count = 1 # print count every 200 photos to monitor progress if count % 200 == 0: print(count)
# Call function to crop full-face eye imageseye_cropper('yourfullfaceimagefolder') 以下是我們用來訓練模型的數(shù)據(jù)示例：
創(chuàng)建卷積神經(jīng)網(wǎng)絡確定指標因為預測正面類別（熟睡的司機）對我們來說比預測負面類別（清醒的司機）更重要，所以我們最重要的指標是召回率（敏感性）。召回率越高，模型錯誤地預測清醒（假陰性）的睡眠驅(qū)動程序的數(shù)量就越少。這里唯一的問題是我們的正面類別明顯多于我們的負面類別。因此，最好使用F1 分數(shù)或 Precision-Recall AUC 分數(shù)，因為它們還考慮了我們猜測駕駛員睡著但實際上清醒（精確）的次數(shù)。否則我們的模型將始終預測我們處于睡眠狀態(tài)，無法使用。另一種我們在處理不平衡圖像數(shù)據(jù)時沒有使用的方法是使用圖像增強，我沒有在這里使用它，但是 Jason Brownlee 在解釋如何在這里使用它方面做得很好。準備圖像數(shù)據(jù)下一步是導入圖像并用模型進行預處理。本節(jié)所需的導入：
count = 1 if count % 500 == 0: print('Succesful Image Import Count = ' str(count))
return images
folder="../data/train/new_open_eyes"open_eyes = load_images_from_folder(folder, 0)
folder="../data/train/New_Closed_Eyes"closed_eyes = load_images_from_folder(folder, 1)eyes = close_eyes open_eyes 設置變量，獨立的 X 是圖像，依賴的 y 是相應的標簽（1 表示閉眼，0 表示睜眼）：
卷積層：該層創(chuàng)建像素子集而不是完整圖像，并允許更快的模型。根據(jù)設置的過濾器數(shù)量，這可能比原始圖像的密度更高或更低，但它們將使模型能夠使用更少的資源了解更復雜的關系。我們使用 32 個過濾器，使用至少一個卷積層，通常需要兩個或更多。對我們來說，最佳設置是將兩個3x3組合在一起，然后將三個3x3組合在一起。CNN 的總體趨勢是使用較小的濾波器尺寸。事實上，雙3x3層與5x5層基本相同，但速度更快，通常會產(chǎn)生更好的分數(shù)。壓平確保展平圖像陣列，以便它可以進入密集層。密集層層越密集，我們的模型訓練所需的時間就越長，隨著這些層中神經(jīng)元數(shù)量的增加，網(wǎng)絡學習到的關系的復雜性也會增加。一般來說，通常卷積層的想法是為了避免產(chǎn)生過深的密集層方案。在我們的模型中我們使用了三層，神經(jīng)元的relu激活率呈下降趨勢（256、128、64）。我們還在每一層之后使用了 30% 的dropout。輸出層最后，因為這是一個二進制分類問題，請確保對外層使用 sigmoid 激活。編譯模型在 model.compile()中，我們需要將指標設置為 PR AUC（tf.keras.metrics.AUC (curve = 'PR')在 tensorflow 中）或召回率（tf.keras.metrics.recall在 tensorflow 中）。將損失設置為二進制交叉熵，因為這通常是一個二進制分類模型和一個好的優(yōu)化器。擬合模型將批量大小設置得盡可能大。我在 Google Colab 的 32 GB TPU 上運行了 gridsearch，它輕松運行了 1000 多個批次。如有疑問，請嘗試 32 個批次，如果沒有使內(nèi)存過載，則增加。就epochs 而言，20 個 epochs 后收益遞減，所以我不會比這個特定的 CNN 高太多。以下是 Tensorflow Keras 的完整設置：
# Adding first three convolutional layersmodel.add(Conv2D( filters = 32, # number of filters kernel_size = (3,3), # height/width of filter activation = 'relu', # activation function input_shape = (80,80,3) # shape of input (image) ))model.add(Conv2D( filters = 32, # number of filters kernel_size = (3,3), # height/width of filter activation = 'relu' # activation function ))model.add(Conv2D( filters = 32, # number of filters kernel_size = (3,3), # height/width of filter activation = 'relu' # activation function ))
# Adding pooling after convolutional layersmodel.add(MaxPooling2D(pool_size = (2,2))) # Dimensions of the region that you are pooling
# Adding second set of convolutional layersmodel.add(Conv2D( filters = 32, # number of filters kernel_size = (3,3), # height/width of filter activation = 'relu' # activation function ))model.add(Conv2D( filters = 32, # number of filters kernel_size = (3,3), # height/width of filter activation = 'relu' # activation function ))
# Add last pooling layer.model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Flatten())
# Adding first dense layer with 256 nodesmodel.add(Dense(256, activation='relu'))
# Adding a dropout layer to avoid overfittingmodel.add(Dropout(0.3))
model.add(Dense(128, activation='relu'))model.add(Dropout(0.3))
model.add(Dense(64, activation='relu'))model.add(Dropout(0.3))
# adding output layermodel.add(Dense(1, activation = 'sigmoid'))
# compiling the modelmodel.compile(loss='binary_crossentropy', optimizer='adam', metrics=[tf.keras.metrics.AUC(curve = 'PR')])
# fitting the modelmodel.fit(X_train, y_train, batch_size=800, validation_data=(X_test, y_test), epochs=24)
# evaluate the model model.evaluate(X_test, y_test, verbose=1) 曲線下的最終精確召回區(qū)域：0.981033創(chuàng)建網(wǎng)絡攝像頭應用程序獲得滿意的模型后，請使用model.save('yourmodelname.h5'). 保存生產(chǎn)模型時，請確保運行該模型時沒有驗證數(shù)據(jù)，這將在導入時導致問題。安裝和導入：這些是 Mac 優(yōu)化的，盡管也可以在 Windows 上使用相同的腳本。
else: counter = counter 1 status = ‘Closed’ cv2.putText(frame, status, (round(w/2)-104,70), cv2.FONT_HERSHEY_SIMPLEX, 2, (0,0,255), 2, cv2.LINE_4) 如果一行中有6幀是閉著眼睛的（“睡眠”），我們還希望顯示一個警報。這可以使用一個簡單的if語句來完成：
最后，我們需要顯示幀并為 while 循環(huán)提供退出鍵。在cv2.waitKey(1)確定幀的顯示時間。括號中的數(shù)字是幀將顯示的毫秒數(shù)，除非按下“k”鍵（在本例中為27）或escape鍵：
?正如我們所見，該模型非常有效，盡管訓練時間很長，但仍可在幾毫秒內(nèi)返回預測。通過一些進一步的改進并導出到外部機器，這個程序可以很容易地應用于實際情況，甚至可以挽救生命。
# webcam frame is inputted into functiondef eye_cropper(frame):
# create a variable for the facial feature coordinates facial_features_list = face_recognition.face_landmarks(frame)
# create a placeholder list for the eye coordinates # and append coordinates for eyes to list unless eyes # weren't found by facial recognition try: eye = facial_features_list[0]['left_eye'] except: try: eye = facial_features_list[0]['right_eye'] except: return
# establish the max x and y coordinates of the eye x_max = max([coordinate[0] for coordinate in eye]) x_min = min([coordinate[0] for coordinate in eye]) y_max = max([coordinate[1] for coordinate in eye]) y_min = min([coordinate[1] for coordinate in eye])
# establish the range of x and y coordinates x_range = x_max - x_min y_range = y_max - y_min
# in order to make sure the full eye is captured, # calculate the coordinates of a square that has a # 50% cushion added to the axis with a larger range and # then match the smaller range to the cushioned larger range if x_range > y_range: right = round(.5*x_range) x_max left = x_min - round(.5*x_range) bottom = round((((right-left) - y_range))/2) y_max top = y_min - round((((right-left) - y_range))/2) else: bottom = round(.5*y_range) y_max top = y_min - round(.5*y_range) right = round((((bottom-top) - x_range))/2) x_max left = x_min - round((((bottom-top) - x_range))/2)
# crop the image according to the coordinates determined above cropped = frame[top:(bottom 1), left:(right 1)]
# resize the image cropped = cv2.resize(cropped, (80,80)) image_for_prediction = cropped.reshape(-1, 80, 80, 3)
return image_for_prediction
# initiate webcamcap = cv2.VideoCapture(0)w = cap.get(cv2.CAP_PROP_FRAME_WIDTH)h = cap.get(cv2.CAP_PROP_FRAME_HEIGHT)if not cap.isOpened(): raise IOError('Cannot open webcam')
# set a countercounter = 0
# create a while loop that runs while webcam is in usewhile True:
# capture frames being outputted by webcam ret, frame = cap.read()
# use only every other frame to manage speed and memory usage frame_count = 0 if frame_count == 0: frame_count = 1 pass else: count = 0 continue
# function called on the frame image_for_prediction = eye_cropper(frame) try: image_for_prediction = image_for_prediction/255.0 except: continue
# get prediction from model prediction = eye_model.predict(image_for_prediction)
# Based on prediction, display either "Open Eyes" or "Closed Eyes" if prediction < 0.5: counter = 0 status = 'Open'
cv2.rectangle(frame, (round(w/2) - 110,20), (round(w/2) 110, 80), (38,38,38), -1)
cv2.putText(frame, status, (round(w/2)-80,70), cv2.FONT_HERSHEY_SIMPLEX, 2, (0,255,0), 2, cv2.LINE_4) x1, y1,w1,h1 = 0,0,175,75 ## Draw black backgroun rectangle cv2.rectangle(frame, (x1,x1), (x1 w1-20, y1 h1-20), (0,0,0), -1) ## Add text cv2.putText(frame, 'Active', (x1 int(w1/10), y1 int(h1/2)), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 255,0),2) else: counter = counter 1 status = 'Closed'
cv2.rectangle(frame, (round(w/2) - 110,20), (round(w/2) 110, 80), (38,38,38), -1)
cv2.putText(frame, status, (round(w/2)-104,70), cv2.FONT_HERSHEY_SIMPLEX, 2, (0,0,255), 2, cv2.LINE_4) x1, y1,w1,h1 = 0,0,175,75 ## Draw black backgroun rectangle cv2.rectangle(frame, (x1,x1), (x1 w1-20, y1 h1-20), (0,0,0), -1) ## Add text cv2.putText(frame, 'Active', (x1 int(w1/10), y1 int(h1/2)), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 255,0),2)
# if the counter is greater than 3, play and show alert that user is asleep if counter > 2:
## Draw black background rectangle cv2.rectangle(frame, (round(w/2) - 160, round(h) - 200), (round(w/2) 160, round(h) - 120), (0,0,255), -1) cv2.putText(frame, 'DRIVER SLEEPING', (round(w/2)-136,round(h) - 146), cv2.FONT_HERSHEY_SIMPLEX, 1, (0,0,0), 2, cv2.LINE_4) cv2.imshow('Drowsiness Detection', frame) k = cv2.waitKey(1) ## Sound playsound('rooster.mov') counter = 1 continue
cv2.imshow('Drowsiness Detection', frame) k = cv2.waitKey(1) if k == 27: breakcap.release()cv2.destroyAllWindows()
下載文檔
