CatalystBERT-v1

Test results by reaction horizon

Impact significance ranks whether a reaction is large relative to expected volatility. Direction ranks positive versus negative returns among significant reactions. These are separate tasks.

Impact ranking is strongest at shorter horizons; direction remains near chance. The aggregate scores do not show calibration, confidence intervals, or subgroup performance.

Data & methodology

One example represents an article–target-company pair. The same article can have different implications for different companies. Publication timing determines which return windows can be labeled; future prices create training targets and are never inference inputs.

Article sources and timing

The current builder defaults to Benzinga, Finnhub, and Yahoo. Historical Google News is opt-in because manual review found false target matches for common-word tickers.

Intraday endpoints use the latest completed minute-bar close, rejecting endpoint prices more than five minutes stale. Daily volatility is the sample standard deviation of up to 20 prior daily stock-minus-SPY log returns, requiring at least 10 valid observations.

Connecting article labels to market returns

Horizon minutes are [5, 30, 120, 390]. A scaled return of +1 means a positive abnormal move of one expected volatility, not a 1% price return. Missing or nonpositive expected volatility makes a target unavailable. SPY adjustment removes its observed log return; this is not a beta-adjusted asset-pricing alpha estimate.

Label quality and leakage controls

• Mask a horizon if another article for the same asset falls inside its reaction window.
• Mask 1d targets crossing a material price-adjustment factor change.
• By default, exclude 09:30–10:00 ET publications from impact training and mask horizons whose prior move reaches three expected volatilities.
• Carry a separate target mask per horizon. Missing targets are zero-filled for batching but excluded from losses and evaluation.
• Use chronological splits and an asset map derived only from final training rows. Unseen validation, test, and inference assets map to 0.

These controls reduce contamination; they do not prove causal attribution to an article. The cleaned pipeline has evolved since the reported experiment and still needs controlled reproduction.

Input and output tensors

The tables document the current combined-mode implementation associated with this v1 research. B is batch size; L is padded token length; F = 5 market features; the four horizon columns always follow [5m, 30m, 2h, 1d]. The reported experiment used batch size 1 and maximum L = 1,024. An archived checkpoint’s own configuration remains authoritative.

Text serialization

[TARGET] Target Company (SYMBOL)
[TIME] datetime | date
[TITLE] Article headline
[CONTENT] Article body…

The prefix and article body are tokenized as a pair. Truncation applies only to the content sequence, preserving target and title. Four section tokens are added to the tokenizer and the encoder embedding vocabulary is resized accordingly. No token_type_ids tensor is passed.

Market-feature order

1. pre_event_abnormal_return_5m
2. pre_event_abnormal_return_30m
3. pre_event_abnormal_return_2h
4. pre_event_abnormal_return_1d
5. pre_event_volatility_20d

The 1d pre-event feature uses the prior market close; date-only articles use strictly prior daily context and have no intraday context. The feature vector and its availability mask are concatenated to [B, 10] before the market encoder.

Forward computation can use mixed precision; the inference adapter explicitly casts outputs to float32 before sigmoid and magnitude decoding. Sigmoid scores are not established calibrated probabilities. Magnitude is an absolute reaction in volatility units, not a direct price forecast.

Training targets and losses

ModernBERT-large

A bidirectional, encoder-only Transformer used here to encode article text, rather than generate text. ModernBERT-large has 395 million parameters and 28 layers, with a native context window up to 8,192 tokens. It uses rotary positional embeddings, alternating local/global attention, and unpadding; this implementation requests FlashAttention 2. Official model card.

The encoder configuration specifies a hidden width of 1,024. CatalystBERT mean-pools non-padding token embeddings and adds its own structured inputs and prediction heads. The experiment’s 1,024-token cap is substantially shorter than the encoder’s supported context window. These are distinct limits.

Reported experiment and limitations

catalystbert-expanded-v1-combined: 95,001 training, 30,445 validation, and 30,947 test examples. Three epochs; batch size 1; eight gradient-accumulation steps; learning rate 2e-5; seed 17. The checkpoint with lowest validation loss was evaluated on the chronological test split.

These aggregate ablations suggest a small combined-input improvement in the documented experiment; no confidence interval or significance test is available. The model’s direction scores do not support a directional trading claim. Magnitude quality, calibration, and downstream strategy value are not established by the available results.