A faster way to search through files than grepping?

Question

Is there a way to index a file or a file tree from the console so that searching for substrings can be done faster than plain grepping?

I don't necessarily need support for regular expressions, although case insensitivity would be good to have.

I'm looking for a simple console utility that would work similar to locate/slocate/mlocate but instead of file names would index file contents with specific mime types in a configurable location.

So far the only 'faster-than-grep' solution which I've found is fgrep constrained to ANSI rather than UTF-8 (from https://stackoverflow.com/a/13913220/191246) — while it provides an impressive speedup, it is still too slow for large files. I would like to know if there is some 'cheap' way to create an index and search against it.

I am considering whoosh as an option but that would require extra coding.

I am not interested in system-level indexing apps like spotlight on mac or their linux counterparts, since I am looking for something granular at file or subfolder level.

Answer 1

Other alternatives which don't require an index, include:

ripgrep https://blog.burntsushi.net/ripgrep/

ag aka the silver searcher: https://geoff.greer.fm/ag/

ack https://beyondgrep.com/

Answer 2

Google code search command line utilties (written in Go) fit the described use case.

On debian/ubuntu it can be installed with

sudo apt install codesearch

To compile from source:

If not already present, install first the golang environment, and define the GOPATH variable:

sudo mkdir -p /usr/local/go
sudo chown myusername /usr/local/go

## normally you'd put this in your ~/.bashrc
export GOPATH=/usr/local/go

## on ubuntu/debian:
sudo apt install golang

## on osx
brew install golang

Next, build cindex and csearch:

go get github.com/google/codesearch/cmd/...

Once installed from source you will have cindex and csearch under your $GOPATH/bin — either move them to somewhere under your $PATH, or add $GOPATH/bin to your $PATH and refresh/restart your shell.

Usage

Index a file or folder:

cindex myproject_dir

The index will be created in ~/.csearchindex.

Now you can search the index:

csearch sausage-and-spam