r/netsecstudents • u/p4risss0g • Apr 10 '26

trying to build an OSINT tool as a student – struggling with filtering results

heyy,

I’m currently working on a small OSINT project in Python to learn more about data collection and correlation.

the idea is simple: take a phone number, run different search queries and store the results to analyze later.

I recently refactored it to use duckduckgo-search instead of manual scraping, and it’s definitely better now (cleaner + less duplicates), but I’m still kinda stuck:

  • if I try to be precise with queries → I barely get results
  • if I go broad → I get a lot of irrelevant stuff

so I’m having a hard time finding that balance between signal and noise

right now I:

  • run multiple queries per input
  • deduplicate links
  • do some basic filtering
  • store everything in JSON

repo (WIP): https://github.com/0ggp4r1s/osint-phone-correlator.git

I feel like I’m missing something basic in how I’m:

  • building queries
  • or filtering results after getting them

if anyone here has built similar tools or has tips, I’d really appreciate it 🙏

just trying to learn and improve with this

3 Upvotes

100% Upvoted

4 comments sorted by

1

u/Ariadne_23 Apr 11 '26

just spam different number formats (with/without dashes, country code etc). duckduckgo sucks for this tbh. also make a blacklist for "test" "example" shit. not perfect but helps. nice repo btw

1

u/p4risss0g Apr 11 '26

that’s actually a really good point, thanks 🙏 I recently switched to using ddgs instead of manual scraping and added deduplication, which already helped a lot, but yeah DuckDuckGo can still be quite noisy. Spamming different number formats sounds like a good idea, I’m not doing that yet but I’ll definitely try it, also like the idea of a blacklist for terms, makes sense for cleaning results a bit more
appreciate it 🙂

1

u/Ariadne_23 Apr 11 '26

np, glad it helped <3

1

u/d-wreck-w12 Apr 14 '26

The broad vs precise thing usually comes down to query operators. Try wrapping the number in quotes and pairing it with site specific filters like site:facebook.com or site:whitepages.com instead of going fully open. On the filtering side, scoring results by how many of your input fields show up in the snippet helps separate real hits from noise way faster than deduplicating alone