Cybersecurity

6372 readers

31 users here now

c/cybersecurity is a community centered on the cybersecurity and information security profession. You can come here to discuss news, post something interesting, or just chat with others.

THE RULES

Instance Rules

Be respectful. Everyone should feel welcome here.
No bigotry - including racism, sexism, ableism, homophobia, transphobia, or xenophobia.
No Ads / Spamming.
No pornography.

Community Rules

Idk, keep it semi-professional?
Nothing illegal. We're all ethical here.
Rules will be added/redefined as necessary.

If you ask someone to hack your "friends" socials you're just going to get banned so don't do that.

Learn about hacking

Hack the Box

Try Hack Me

Pico Capture the flag

Other security-related communities !databreaches@lemmy.zip !netsec@lemmy.world !securitynews@infosec.pub !cybersecurity@infosec.pub !pulse_of_truth@infosec.pub

Notable mention to !cybersecuritymemes@lemmy.world

founded 2 years ago

MODERATORS

kid@sh.itjust.works

Lanky_Pomegranate530@midwest.social

DeepSeek found to be sharing user data with TikTok parent company ByteDance (www.malwarebytes.com)

submitted 2 days ago by kid@sh.itjust.works to c/cybersecurity@sh.itjust.works

28 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] brucethemoose@lemmy.world 2 points 1 day ago* (last edited 1 day ago) (1 children)

There are plenty of free LLM APIs you can use with something like Open Web UI, on any machine. I still use them myself.

[–] jwiggler@sh.itjust.works 1 points 1 day ago (2 children)

Have you got any recs? I've got a 3080 in my machine atm

[–] horse_battery_staple@lemmy.world 2 points 1 day ago

I'm not @brucethemoose@lemmy.world However here's a pretty barebones how to article to get you started. Just know it can be as complicated as you like. For starters you may want to stick to the 7b and 14b models like mistral:7b and phi4:14b as they'll fit easily on your card and will allow you to test the waters.

If you're on Windows https://doncharisma.org/2024/11/23/self-hosting-ollama-with-open-webui-on-windows-a-step-by-step-guide/

If you're using Linux https://linuxtldr.com/setup-ollama-and-open-webui-on-linux/

If you want a container https://github.com/open-webui/open-webui/blob/main/docker-compose.yaml

[–] brucethemoose@lemmy.world 1 points 1 day ago* (last edited 1 day ago)

Locally? Arcee 14B and the 14B Deepseek distill are currently the best models that fill fit.

I'd recommend hosting them with TabbyAPI instead of ollama, as they will be much faster and more VRAM efficient. But this is more fuss.

Honestly, I would just try free APIs like Gemini, Groq, and such through open web ui, or use really cheap APIs like openrouter. Newer 14B models are okay, but they're definitely lacking that "encyclopedic intelligence" larger models have.