Llm-Security on CuraSec

Malicious Free LLM Endpoints Silently Harvesting Coding-Agent Context

Tue, 01 Sep 2026 15:28:52 +0000

Engineer — Learn: Honeypot research shows that untrusted ‘free’ LLM backends receive full coding-agent context — filesystem paths, conversation history, tool manifests — before any response is sent. Audit every LLM endpoint configured in your coding agents and ensure all traffic goes to verified, first-party providers.
SOC/IR — Learn: Demonstrates a passive exfiltration path: coding agents silently send working paths and tool manifests to whatever endpoint they’re pointed at. No IOCs or active campaign here, but useful context for future detections around unexpected outbound HTTPS from dev tools to novel LLM API hosts.
Leader — Plan: Employees using unofficial ‘free’ AI coding tools may be routing sensitive codebase context and filesystem details to unverified third parties; establish or enforce an approved-LLM-provider policy for coding agents this quarter before an incident forces a reactive response.

DP Bounds on LLM Memorization vs. Extraction Are Not Interchangeable

Mon, 31 Aug 2026 19:07:02 +0000

Engineer — Learn: If you rely on DP guarantees to protect training data in ML pipelines, this research shows that controlling memorization and controlling extraction are formally separate — a model can be memorized yet unextractable, or vice versa. Revisit your threat model assumptions, but no system change is required today.
SOC/IR — Skip
Leader — Skip

FISGuard: Defense Against Membership Inference in Federated LLMs

Mon, 31 Aug 2026 19:07:02 +0000

Engineer — Learn: Novel defense technique for federated fine-tuning pipelines; relevant if you run distributed LLM training with sensitive data, but no patch or configuration action needed today — research-stage only.
SOC/IR — Skip
Leader — Skip

Mechanistic Interpretability Study Maps LLM Jailbreak Circuits

Mon, 31 Aug 2026 19:07:02 +0000

Engineer — Learn: Research identifies internal attention heads and MLP pathways responsible for safety bypass in LLaMA-2-7B — useful context when evaluating LLM safeguard architectures, but no operational change needed today and findings are on one specific model.
SOC/IR — Skip
Leader — Learn: Findings suggest current LLM safety alignment has exploitable structural weaknesses; relevant context when assessing risk posture of internally deployed LLM products, but no immediate action required.

Meta-Adaptive Jailbreaking Achieves 80%+ ASR Against Frontier VLMs

Mon, 31 Aug 2026 19:07:02 +0000

Engineer — Learn: If your product integrates GPT-4o, Gemini, or similar multimodal models, this research shows existing content-safety wrappers are brittle against adaptive attackers; no patch exists yet, but it motivates evaluating your VLM endpoints against adaptive prompt-injection test suites.
SOC/IR — Skip
Leader — Learn: Research demonstrating high-success jailbreaks against GPT-4o and Gemini is useful framing for board-level AI risk discussions and for questioning AI vendor safety assurance claims when procuring or expanding VLM-based tooling.

zk-SNARK Adversarial Probes Detect Post-Deployment LLM Tampering

Mon, 31 Aug 2026 19:07:02 +0000

Engineer — Learn: Academic framework for detecting model drift after deployment using privacy-preserving proofs; no running systems to patch today, but the black-box token-probe approach is worth tracking as LLM supply-chain integrity tooling matures.
SOC/IR — Skip
Leader — Learn: Offers a governance-relevant framing: proprietary LLMs can be silently altered post-approval, and cryptographic audit frameworks are emerging to address that gap — useful context for AI risk discussions with the board or auditors.

Research: LLM Safety Refusals Are Fragile and Easily Bypassed

Sat, 29 Aug 2026 15:36:18 +0000

Engineer — Learn: Reinforces the design principle that LLM safety filters alone are insufficient; architecture decisions should place external guardrails (input/output validation, prompt firewalls) outside the model layer rather than trusting built-in refusals.
SOC/IR — Skip
Leader — Learn: Supports the case for defense-in-depth policy around AI deployments: if safety refusals are fragile by design, any AI system handling sensitive data needs external controls beyond the model’s built-in guardrails — useful framing for board or audit conversations about AI risk.

aiXamine: Cross-dimensional LLM safety/security/privacy evaluation study

Mon, 24 Aug 2026 13:10:29 +0000

Engineer — Learn: The findings — that safety alignment increases over-refusal (safety tax), privacy is near-orthogonal to other trustworthiness dimensions, and distillation degrades robustness — are useful mental models for engineers selecting or evaluating LLMs in their stack, though no immediate system changes are required.
SOC/IR — Skip
Leader — Learn: The finding that strong alignment does not protect privacy, and that distilled models suffer robustness collapse, provides empirical grounding for AI governance decisions and risk conversations with leadership about LLM adoption — useful for future board decks but no same-week action needed.

CyberLLM: Multi-Agent LLM Framework for Automotive SDV Security

Mon, 10 Aug 2026 13:39:41 +0000

Engineer — Learn: The hybrid deterministic-plus-LLM pipeline (regex/AST/topology plus LLM refinement) that roughly doubles vulnerability coverage over static rules alone is worth tracking as a design pattern for AppSec tooling, though the automotive ECU focus makes it directly applicable only in that niche.
SOC/IR — Skip
Leader — Skip

LoRAScan: Runtime detection of backdoored LoRA adapters via activation spikes

Mon, 10 Aug 2026 13:39:41 +0000

Engineer — Learn: Identifies a real supply-chain risk for teams consuming third-party LoRA adapters: a backdoored adapter can alter model output on hidden triggers without modifying base model weights. LoRAScan’s inference-time monitoring approach is worth evaluating if your ML pipelines pull adapters from untrusted registries or Hugging Face.
SOC/IR — Learn: No active exploitation, IOCs, or ATT&CK-mappable TTPs to act on; this is foundational research on a threat class. Worth filing as context if your org is building detections around AI/ML pipeline integrity, but no hunt or rule work warranted today.
Leader — Learn: Surfaces an emerging supply-chain risk category for AI workloads—untrusted fine-tuned adapters as a malware vector—useful background for shaping AI vendor-risk policy before it becomes a control requirement.

Policy-Masked Private Experts: Access Control in Sparse MoE LLMs

Mon, 10 Aug 2026 13:39:41 +0000

Engineer — Learn: Novel architecture for parameter-level capability gating in MoE models (tested on Qwen3-30B and DeepSeek-V2-Lite); worth tracking if your team deploys or fine-tunes MoE-based models and needs verifiable separation between capability tiers — no production tooling yet.
SOC/IR — Skip
Leader — Skip

Prompt Injection via 'Ask AI' Deep Links Silently Poisons LLM Memory

Thu, 06 Aug 2026 13:03:19 +0000

Engineer — Learn: Novel attack class: hidden payloads in pre-filled AI deep links can alter LLM memory without user awareness. No exploitation signals or PoC, but engineers building AI-integrated features should audit any ‘Ask AI’ button implementations for unsanitized prompt passthrough.
SOC/IR — Learn: No IOCs, ATT&CK mapping, or active campaign indicators are present. Worth tracking as AI assistant adoption grows, but there is no detection surface or hunt query to act on today.
Leader — Plan: This attack class is relevant to any enterprise deploying AI assistants with memory or context features; factor it into AI acceptable-use policy and vendor evaluation criteria before broader rollout.

GUI Agent Guardrails Erode Under Multi-Turn User Persuasion

Mon, 03 Aug 2026 15:12:30 +0000

Engineer — Learn: Research shows that single-turn ASR benchmarks overstate real-world robustness of GUI agent guardrails, with 4-turn escalation chains recovering ~20 points of attack success across all tested models. Teams building or deploying GUI agents should treat static prompt-level alignment as insufficient and evaluate multi-turn threat scenarios in their safety testing.
SOC/IR — Skip
Leader — Learn: If your organization is piloting or deploying AI GUI agents, this research illustrates that current safety guardrails are weaker than benchmark numbers suggest under realistic multi-turn user interaction — useful context for AI deployment policies and vendor capability reviews, but no immediate action required.

ADSD Attack Collapses Speculative Decoding Acceptance, Slows LLM Inference 62%

Mon, 27 Jul 2026 15:10:27 +0000

Engineer — Learn: Novel prompt-suffix attack degrades speculative decoding throughput without corrupting outputs, affecting any deployment using draft-target inference acceleration (vLLM, TGI, etc.). No patch or mitigation exists yet; file this when designing LLM serving infrastructure to justify input validation and rate controls at the prompt layer.
SOC/IR — Skip
Leader — Skip

LLM + Code Slicing for NFT Smart Contract Vulnerability Detection

Mon, 27 Jul 2026 15:10:27 +0000

Engineer — Learn: Interesting research combining code slicing with LLM analysis to detect reentrancy and overflow in ERC-721 contracts, but no tooling release or actionable change to running systems today.
SOC/IR — Skip
Leader — Skip

Prompt syntax shapes secure code output in open LLMs

Mon, 20 Jul 2026 14:31:24 +0000

Engineer — Learn: Research shows specific syntactic elements (constraints, guards, conditions) placed in prompts consistently reduce insecure code generation from open LLMs — useful input for teams building internal coding assistants or prompt templates for developer tooling.
SOC/IR — Skip
Leader — Skip

Transferable CoT Jailbreaks Bypass LLM Output Safeguards at Scale

Mon, 20 Jul 2026 14:31:24 +0000

Engineer — Learn: Research shows output-only filters like Llama-Guard 3 are insufficient against reasoning-layer attacks; teams building AI applications should evaluate reasoning context, not just final outputs, when designing safety architectures.
SOC/IR — Skip
Leader — Learn: Finding that reasoning-capable models are 2x+ more vulnerable and standard output safeguards regularly fail has implications for enterprise AI risk posture; useful context for AI usage policies and vendor safety attestation reviews.

Malaika: LLM Multi-Agent Framework for Android Malware Behavior Analysis

Mon, 13 Jul 2026 14:30:14 +0000

Engineer — Learn: Academic research on grounded agentic reasoning for malware behavior reconstruction; no immediate engineering action, but the tri-grounding approach (domain, semantics, knowledge) is worth noting when evaluating LLM-assisted code-analysis tooling.
SOC/IR — Learn: Malaika’s behavior-reconstruction framing — connecting sparse program evidence to auditable behavioral conclusions — could inform how teams structure LLM-assisted malware triage workflows, though no detection or hunt action is available from this paper alone.
Leader — Skip

SeedSmith: LLM-Agentic Seed Synthesis for Directed Fuzzing

Mon, 13 Jul 2026 14:30:14 +0000

Engineer — Learn: Novel research showing an LLM-agentic pipeline that improves directed fuzzer crash-trigger rates by generating semantically aware seed corpora; worth evaluating if your team runs fuzzing campaigns against internal C/C++ codebases, but no immediate change to running systems is required.
SOC/IR — Skip
Leader — Skip

SherAgent: LLM-Powered Provenance Graph Attack Investigation

Mon, 13 Jul 2026 14:30:14 +0000

Engineer — Skip
SOC/IR — Learn: SherAgent demonstrates a 31–64% improvement in automated attack investigation success rates using LLM-driven provenance graph backtracking — useful context for teams evaluating or building AI-assisted triage workflows, though no production tool or IOCs are released here.
Leader — Learn: Research from a real SOC environment shows LLM-assisted alert triage meaningfully reduces the manual investigation backlog; relevant background for leaders assessing AI tooling investments in detection and response.

SLBench: LLM Agents Fail Skill Logical Constraints at 70% Rate

Mon, 13 Jul 2026 14:30:14 +0000

Engineer — Learn: If you deploy LLM agents with skill files or tool orchestration, this research quantifies a real risk class: agents routinely violate preconditions and constraints, producing privacy leaks and unsafe config changes. No patch action today, but the SLGuard scaffold approach is worth evaluating if you build skill-guided agents.
SOC/IR — Skip
Leader — Learn: Academic evidence that LLM agents fail safety constraints at high rates is useful background for AI governance discussions, but there is no immediate vendor exposure or regulatory trigger here — file for the next AI risk policy review.