소스 검색

feat: initial sketch of WhisperFE (WFE)

WhisperFE will be slightly different to other FEs in that there is/are specific
target words to be searched for. Not specifying these could be an error (this
commit specifies this as such) but a better approach may be to downgrade that to
a (logging) notice, and simply match nothing / early exit.
main
Rob Hallam 2 달 전
부모
커밋
66b6a4abdc
1개의 변경된 파일37개의 추가작업 그리고 0개의 파일을 삭제
  1. +37
    -0
      pipeline/feature_extractors.py

+ 37
- 0
pipeline/feature_extractors.py 파일 보기

@@ -400,3 +400,40 @@ class JSONFeatureExtractor(FeatureExtractor):

def teardown(self):
pass


class WordFeatureExtractor(FeatureExtractor):
"""Feature extractor for specific word detection (uses Whisper)"""
def __init__(self, input_files=None, config=None):
if not input_files:
raise ValueError("No input files provided!")
self.input_files = input_files
self.config = config
self.features = []

def setup(self, words=[]):
"""Setup the word feature extractor -- validate input files & config

Whisper expects a list of words to search for in the audio
"""
logger.debug("WordFeatureExtractor setup")

# Validate input files
if not self.input_files:
raise ValueError("No input files provided")

# Validate words
if not words:
raise ValueError("No words provided")
words = words
# TODO: consider stripping punctuation since Whisper produces words+punctuation
# and we might want to strip the punctuation there too

def run(self):
"""Extract features corresponding to supplied target words (defined in setup) for each input file

Use Whisper to detect words in the audio, then match these to target words and create features
"""

def teardown(self):
"""Clean up after Whisper"""

불러오는 중...
취소
저장