diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..bc74670 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,115 @@ +# Changelog + +## 0.3.0 (unreleased) + +A correctness review and breaking refactor. Old names are **not** kept as +aliases; use the tables below to migrate. + +### Behavior changes + +- **All information quantities are in bits**, including log-likelihoods + (`log_likelihood`, the Baum-Welch trace, cross-validated log-likelihood, + HDP-HMM traces), topological entropy (was nats), and collision entropy (was + nats). AIC, AICc, BIC, and WAIC keep their standard deviance scale (computed + from the natural log-likelihood); the MDL code length is in bits. `score` and + `observed_information` remain derivatives of the natural log-likelihood, so + standard errors are unchanged. +- **Covers follow Lind & Marcus**: *right* means right-resolving. + `RightFischerCover` is now the exact minimal right-resolving presentation + (it was built from follower sets truncated at length 8 and labeled *left*); + reducible shifts raise `SoficValidationError`. `LeftFischerCover` is its + mirror image. +- `MaximizedPrimeAtomaton` subclasses `NFA`, not `AtomicAutomaton`: it is the + reverse of the canonical RFSA of the reversed language and need not be atomic. +- `CanonicalVisiblyPushdownAutomaton.from_vpa` raises + `NonWellMatchedLanguageError` for languages with pending calls or returns. +- `DeterministicVisiblyPushdownAutomaton.from_vpa` determinizes + nondeterministic input instead of raising. +- VPA boolean and structural operations return concrete automata. +- Wildcard VPA returns fire on every stack symbol and, with a bottom symbol, on + the empty stack (the simulator's behavior; the docstring said otherwise). +- `ShiftOfFiniteType.from_forbidden_words` raises instead of silently + truncating at `max_states`; an empty forbidden set builds the full shift. +- `equivalent()` compares automata over the union of both alphabets. +- `MarkovChain.words_of_length(0)` and `sample_path` respect the initial + distribution; sampling from an initial law with no mass raises `ValueError`. +- `block_entropy_estimates(use_exact=True)` reports crypticity as `C_mu - E`. +- `SlidingBlockCode.apply` keeps constraints longer than the window and raises + when the block map misses an allowed block. +- Baum-Welch raises when every sequence is impossible and warns when some are. +- `BuchiAutomaton.accepts_lasso` rejects an empty loop. +- Stack CSSR no longer drops histories during determinization (which produced + zero-mass states and validation errors). +- TikZ labels: fixed uncompilable `\midcall` / `\uparrowA` and escaped + `\times` / sympy LaTeX. +- cmpy example factories validate parameters like their curated counterparts + (e.g. a coin bias of 1 raises). Processes previously built by `Even`, `Nemo`, + and `ABC` now come from the curated functions and emit integer symbols `0`/`1`; + `alternating_biased_coins(p, p)` keeps its two-state presentation. +- `joint_block_distribution(block_length=n)` counts blocks of `n` symbols + (`history_length=h` meant `h + 1` symbols); the default `2` is unchanged. + +### New + +- Exact canonical RFSA (`CanonicalRFSA.from_language`), maximized prime + átomaton, `CanonicalRFSA.dual` / `MaximizedPrimeAtomaton.dual`, exact + `prime_residuals`, `atoms`, `prime_atoms`, and `ResidualTable`. +- NL\*: `learn_rfsa_nlstar`, `learn_prime_atomaton_nlstar`, + `learn_rfsa_from_language`, and `AutomatonEquivalenceOracle`. +- VPA: `determinize`, `is_empty`, `accepted_word`, `is_universal`, `includes`, + `equivalent`, `has_unmatched_word`; `sofic.automata.vpa.to_single_entry` / + `to_multiple_entry`; modular `minimize` converts automatically when no + modules are given. `NestedWordAutomaton` gains the same operations. +- Exact left/right Krieger covers. +- `sofic.generators.matrices` (joint matrices and start-vector policies) and + `sofic.generators.sampling`. + +### Module moves + +| Old module | New module | +|---|---| +| `sofic.generators.hmm_inference` | `sofic.inference.hmm` (`filtering`, `em`, `information`); `sample` → `sofic.generators.sampling` | +| `sofic.generators.epsilon_inference` | `sofic.inference.cssr` (`process`, `subtree`, `counts`, `significance`); `spectral` → `sofic.inference.spectral` | +| `sofic.generators.epsilon_transducer_inference` | `sofic.inference.cssr.transducer` | +| `sofic.generators.stack_inference` | `sofic.inference.cssr.stack` | +| `sofic.automata.{active,rpni,edsm,dfasat,alergia,papni,observation}` | `sofic.automata.learning.*` | +| `sofic.automata.learning` (NL\*) | `sofic.automata.learning.nlstar` | +| `sofic.automata.vpa`, `vpa_simulation` | `sofic.automata.vpa` package (`base`, `operations`, `deterministic`, `modular`, `canonical`, `simulation`) | +| `sofic.automata.{icdfa,idfa,enumeration}` | `sofic.automata.enumeration.{icdfa,idfa,words}` | +| `sofic.automata.{canonical_extraction,rfsa,atomaton,canonical_dual}` | `sofic.automata.canonical.{residual,rfsa,atomaton,dual}` | +| `sofic.shifts.sofic_relation` | `sofic.shifts.product_alphabet_shift` | + +### Renames and removals + +| Old | New | +|---|---| +| `cssr` | `learn_epsilon_machine_cssr` | +| `subtree_merge` | `learn_epsilon_machine_subtree` | +| `spectral` (wrapper) | `learn_epsilon_machine_spectral` | +| `transcssr` | `learn_epsilon_transducer_cssr` | +| `stack_cssr` / `stack_subtree_merge` / `fit_stack_hmm_mle` | `learn_stack_hmm_cssr` / `learn_stack_hmm_subtree` / `learn_stack_hmm_mle` | +| `suggest_lmax` | `suggest_max_history` | +| `Lmax=`, `L=` (CSSR family and subtree learners) | `max_history=` | +| `reconstruction_sweep(lmaxes=)` | `reconstruction_sweep(max_histories=)` | +| `GoodnessOfFit.L`, `goodness_of_fit(L=)` | `block_length` | +| `forward(scaled=)`, `backward(scaled=)` | `normalize=` | +| `joint_block_distribution(history_length=h)` | `joint_block_distribution(block_length=h + 1)` | +| `QuasiStochasticModel.transition_matrices`, `quasi_inference.transition_matrices` | `symbol_matrices` | +| `sofic.generators.words.hmm_*`, `pfa_*`, `quasi_*`, `markov_*` functions | private; use the model methods | +| `cartesian_product_gg` / `cartesian_product_tt` | `generator_product` / `transducer_product` | +| `compose_tt` / `compose_tg` | `compose_transducers` / `compose_transducer_generator` | +| `wnfa_to_wdfa` / `minimum_wdfa` | `determinize_wheeler` / `minimize_wheeler` | +| `papni_encode` / `papni_encode_samples` | `encode_dyck_word` / `encode_dyck_samples` | +| ICDFA helpers `next_flags`, `string_from_flags`, `flags_from_string`, `count_flag_sequences` | `icdfa_next_flags`, `icdfa_string_from_flags`, `icdfa_flags_from_string`, `icdfa_count_flag_sequences` | +| IDFA helpers `string_from_flags`, `extended_flags`, `transition_count` | `idfa_string_from_flags`, `idfa_extended_flags`, `idfa_transition_count` | +| `CallDrivenAutomaton` | `ModularVisiblyPushdownAutomaton` | +| `CompositeVisiblyPushdownAutomaton`, `union_vpa`, `intersection_vpa`, `complement_vpa`, `difference_vpa`, `concat_vpa`, `kleene_star_vpa` | removed; use the VPA methods | +| `LabeledAutomaton.intersect` / `concatenate` / `star` | `intersection` / `concat` / `kleene_star` | +| `learn_maximized_prime_atomaton` | `learn_prime_atomaton_nlstar` (or `learn_rfsa_nlstar`) | +| `SoficRelation`, `to_sofic_relation` | `ProductAlphabetShift`, `to_product_alphabet_shift` | +| cover `from_sofic` | `from_presentation` | +| `{left,right}_{fischer,krieger}_from_sofic` | `{left,right}_{fischer,krieger}_cover` | +| `sofic.from_yaml`, `serialization.from_yaml` | `model_from_yaml` | +| `examples.processes`: `BiasedCoin`, `FairCoin`, `Even`, `Nemo`, `NRPS`, `ABC` | removed: `bernoulli`, `fair_coin`, `even_process`, `nemo_process`, `noisy_random_phase_slip`, `alternating_biased_coins(1 - p, 1 - q)` | +| `GoldenMean` / `Butterfly` / `PSB` | `golden_mean_forbid_00` / `butterfly_two_branch` / `phase_slip_backtrack_cmpy` | +| other PascalCase `examples.processes` factories (e.g. `BitFlip`, `Delay`, `RestrictedGM`, `IrreversibleTwoState`) | snake_case (`bit_flip`, `delay`, `restricted_gm`, `irreversible_two_state`) | diff --git a/README.rst b/README.rst index 9a26a98..8433bbf 100644 --- a/README.rst +++ b/README.rst @@ -171,7 +171,7 @@ read off computational-mechanics quantities: from sofic import EpsilonMachine eps = EpsilonMachine.from_hmm(gm) # minimize an HMM presentation - # eps = EpsilonMachine.from_sequence(data, method="cssr", Lmax=4) # infer + # eps = EpsilonMachine.from_sequence(data, method="cssr", max_history=4) # infer # eps = EpsilonMachine.from_sequence(data, method="spectral", prefix_length=3, rank=2) eps.statistical_complexity() # 0.9183 bits (C_mu) diff --git a/docs/automata/atomaton.rst b/docs/automata/atomaton.rst index e738518..12ba555 100644 --- a/docs/automata/atomaton.rst +++ b/docs/automata/atomaton.rst @@ -1,5 +1,5 @@ .. atomaton.rst -.. py:module:: sofic.automata.atomaton +.. py:module:: sofic.automata.canonical.atomaton ******** Átomaton @@ -29,13 +29,24 @@ the special case where the reverse is deterministic. .. code-block:: python - from sofic.automata.atomaton import Atomaton, atomic_states, is_atomic + from sofic.automata.canonical.atomaton import Atomaton, atomic_states, is_atomic atomaton = Atomaton.from_language(dfa) is_atomic(atomaton) # True atomic_states(nfa) # states whose right language is a union of atoms is_atomic(nfa.reverse()) # iff nfa.determinize() is minimal +Maximized prime átomaton +======================== + +The maximized prime átomaton (:class:`MaximizedPrimeAtomaton`) is the dual of +the canonical RFSA :cite:`MaarandTamm2022`: the reverse of the canonical RFSA of +the reversed language, just as the átomaton is the reverse of the minimal DFA of +the reversed language. Its states are the maximized prime atoms, and the right +language of each lies between its atom and its maximized atom :cite:`Tamm2015`. +Unlike the átomaton it need not be atomic, so it is a plain +:class:`~sofic.automata.nfa.NFA` subclass. + API === @@ -45,3 +56,6 @@ API .. autoclass:: AtomicAutomaton .. autoclass:: Atomaton .. autoclass:: MaximizedPrimeAtomaton + :members: from_language, from_canonical_rfsa, dual + +.. autofunction:: sofic.automata.canonical.residual.maximized_prime_atomaton_from_language diff --git a/docs/automata/dfa.rst b/docs/automata/dfa.rst index 23a0a49..70b4462 100644 --- a/docs/automata/dfa.rst +++ b/docs/automata/dfa.rst @@ -39,4 +39,4 @@ API === .. autoclass:: DFA - :members: add_transition, recognizes, union, intersection, intersect, complement, difference, concat, concatenate, kleene_star, star, minimize, from_nfa + :members: add_transition, recognizes, union, intersection, complement, difference, concat, kleene_star, minimize, from_nfa diff --git a/docs/automata/icdfa.rst b/docs/automata/icdfa.rst index d4c0f95..5e905ab 100644 --- a/docs/automata/icdfa.rst +++ b/docs/automata/icdfa.rst @@ -1,5 +1,5 @@ .. icdfa.rst -.. py:module:: sofic.automata.icdfa +.. py:module:: sofic.automata.enumeration.icdfa ***** ICDFA @@ -23,7 +23,7 @@ API .. autofunction:: count_icdfa .. autofunction:: count_icdfa_empty -.. autofunction:: sofic.automata.idfa.iter_idfa_strings -.. autofunction:: sofic.automata.idfa.rank_idfa_string -.. autofunction:: sofic.automata.idfa.unrank_idfa_string -.. autofunction:: sofic.automata.idfa.count_accessible_idfa +.. autofunction:: sofic.automata.enumeration.idfa.iter_idfa_strings +.. autofunction:: sofic.automata.enumeration.idfa.rank_idfa_string +.. autofunction:: sofic.automata.enumeration.idfa.unrank_idfa_string +.. autofunction:: sofic.automata.enumeration.idfa.count_accessible_idfa diff --git a/docs/automata/learning.rst b/docs/automata/learning.rst index 4c53400..fcb7717 100644 --- a/docs/automata/learning.rst +++ b/docs/automata/learning.rst @@ -1,5 +1,5 @@ .. learning.rst -.. py:module:: sofic.automata.learning +.. py:module:: sofic.automata.learning.nlstar ******** Learning @@ -10,23 +10,33 @@ Learning Active learning (NL\*) ====================== -Active learning of maximized prime átomatons via NL\* with a membership -teacher, following Angluin-style learning and its nondeterministic extension -:cite:`Angluin1987,Bollig2009`: +NL\* :cite:`Bollig2009` extends Angluin's L\* :cite:`Angluin1987` to +nondeterministic automata. It keeps an RFSA-closed, RFSA-consistent observation +table whose prime rows become the hypothesis states, and adds every suffix of a +counterexample as a new experiment. When the equivalence oracle accepts, the +hypothesis is the canonical RFSA of the target (:doc:`rfsa`). Running NL\* on +the reversed target and reversing the result learns the maximized prime +átomaton (:doc:`atomaton`). -.. autofunction:: sofic.automata.learning.learn_maximized_prime_atomaton +:class:`~sofic.automata.learning.active.AutomatonEquivalenceOracle` answers equivalence +queries exactly against a target automaton, returning a shortest +counterexample. + +.. autofunction:: sofic.automata.learning.nlstar.learn_rfsa_nlstar +.. autofunction:: sofic.automata.learning.nlstar.learn_prime_atomaton_nlstar +.. autofunction:: sofic.automata.learning.nlstar.learn_rfsa_from_language Active learning (L\*, TTT, Mealy) ================================= -:mod:`sofic.automata.active` learns an automaton from a *teacher* answering +:mod:`sofic.automata.learning.active` learns an automaton from a *teacher* answering **membership** and **equivalence** queries. It provides Angluin's L\* :cite:`Angluin1987` and a redundancy-free **discrimination-tree** learner in the TTT family :cite:`KearnsVazirani1994,Isberner2014` for :class:`~sofic.automata.dfa.DFA`, plus the Mealy variant of L\* :cite:`Shahbaz2009`; all use Rivest-Schapire counterexample analysis :cite:`RivestSchapire1993`. Oracles adapt sofic models: a -:class:`~sofic.automata.active.LanguageMembershipOracle` wraps any model exposing +:class:`~sofic.automata.learning.active.LanguageMembershipOracle` wraps any model exposing ``recognizes`` / ``__contains__`` (a :class:`~sofic.automata.dfa.DFA`, NFA, átomaton, or ``model.to_support_dfa()`` for a sofic shift or ε-machine), and the equivalence oracles offer bounded-exhaustive or random-walk testing. @@ -43,7 +53,7 @@ equivalence test: .. code-block:: python - from sofic.automata.active import ( + from sofic.automata.learning.active import ( FunctionMembershipOracle, RandomWalkEquivalenceOracle, learn_dfa_lstar, @@ -53,22 +63,23 @@ equivalence test: equivalence = RandomWalkEquivalenceOracle(membership, {"a", "b"}, rng=0) dfa = learn_dfa_lstar({"a", "b"}, membership, equivalence) -.. autofunction:: sofic.automata.active.learn_dfa_lstar -.. autofunction:: sofic.automata.active.learn_dfa_ttt -.. autofunction:: sofic.automata.active.learn_mealy_lstar -.. autofunction:: sofic.automata.active.learn_dfa_from_language -.. autofunction:: sofic.automata.active.learn_mealy_from_transducer +.. autofunction:: sofic.automata.learning.active.learn_dfa_lstar +.. autofunction:: sofic.automata.learning.active.learn_dfa_ttt +.. autofunction:: sofic.automata.learning.active.learn_mealy_lstar +.. autofunction:: sofic.automata.learning.active.learn_dfa_from_language +.. autofunction:: sofic.automata.learning.active.learn_mealy_from_transducer -.. autoclass:: sofic.automata.active.MembershipOracle +.. autoclass:: sofic.automata.learning.active.MembershipOracle :members: -.. autoclass:: sofic.automata.active.EquivalenceOracle +.. autoclass:: sofic.automata.learning.active.EquivalenceOracle :members: -.. autoclass:: sofic.automata.active.LanguageMembershipOracle -.. autoclass:: sofic.automata.active.FunctionMembershipOracle -.. autoclass:: sofic.automata.active.ExhaustiveEquivalenceOracle -.. autoclass:: sofic.automata.active.RandomWalkEquivalenceOracle -.. autoclass:: sofic.automata.active.TransducerOutputOracle -.. autoclass:: sofic.automata.active.MealyExhaustiveEquivalenceOracle +.. autoclass:: sofic.automata.learning.active.LanguageMembershipOracle +.. autoclass:: sofic.automata.learning.active.FunctionMembershipOracle +.. autoclass:: sofic.automata.learning.active.AutomatonEquivalenceOracle +.. autoclass:: sofic.automata.learning.active.ExhaustiveEquivalenceOracle +.. autoclass:: sofic.automata.learning.active.RandomWalkEquivalenceOracle +.. autoclass:: sofic.automata.learning.active.TransducerOutputOracle +.. autoclass:: sofic.automata.learning.active.MealyExhaustiveEquivalenceOracle Passive learning (RPNI) ======================= @@ -84,7 +95,7 @@ consistent with the sample :cite:`Lang1998`: dfa = learn_dfa_rpni(positive=["ab", "abab"], negative=["a", "b"]) dfa.validate() -.. autofunction:: sofic.automata.rpni.learn_dfa_rpni +.. autofunction:: sofic.automata.learning.rpni.learn_dfa_rpni Passive learning (EDSM / blue-fringe) ===================================== @@ -105,12 +116,12 @@ automaton: dfa = learn_dfa_edsm(positive=["a", "aba", "ababa"], negative=["", "b", "ab"]) dfa.validate() -.. autofunction:: sofic.automata.edsm.learn_dfa_edsm +.. autofunction:: sofic.automata.learning.edsm.learn_dfa_edsm Exact minimal DFA (SAT) ======================= -Where RPNI and EDSM are heuristics, :func:`sofic.automata.dfasat.learn_dfa_sat` +Where RPNI and EDSM are heuristics, :func:`sofic.automata.learning.dfasat.learn_dfa_sat` returns the **provably minimal** DFA consistent with the sample. Following Heule & Verwer :cite:`HeuleVerwer2010`, it translates the augmented prefix-tree acceptor into a graph-colouring SAT instance and searches the state count ``k`` @@ -125,7 +136,7 @@ satisfiable ``k``. It requires the optional `python-sat dfa = learn_dfa_sat(positive=["a", "aba", "ababa"], negative=["", "b", "ab"]) dfa.validate() -.. autofunction:: sofic.automata.dfasat.learn_dfa_sat +.. autofunction:: sofic.automata.learning.dfasat.learn_dfa_sat Probabilistic passive learning (ALERGIA) ======================================== @@ -135,7 +146,7 @@ from **unlabeled** positive strings by merging states of a frequency prefix-tree acceptor whenever a Hoeffding-bound test cannot distinguish their transition statistics :cite:`Carrasco1994`. It is the stochastic, unlabeled counterpart of RPNI/EDSM and a state-merging alternative to CSSR -(:func:`sofic.generators.epsilon_inference.cssr`). The compatibility threshold +(:func:`sofic.inference.cssr.process.cssr`). The compatibility threshold ``alpha`` trades off model size against fidelity: smaller ``alpha`` merges more aggressively (fewer states); larger ``alpha`` is more conservative. @@ -148,13 +159,13 @@ aggressively (fewer states); larger ``alpha`` is more conservative. pfa = learn_pfa_alergia(samples, alpha=0.05) pfa.validate() -.. autofunction:: sofic.automata.alergia.learn_pfa_alergia +.. autofunction:: sofic.automata.learning.alergia.learn_pfa_alergia Passive learning (PAPNI) ======================== PAPNI extends passive inference to visibly pushdown languages. Words over a -:class:`~sofic.automata.papni.DyckAlphabet` are stack-encoded, a DFA is +:class:`~sofic.automata.learning.papni.DyckAlphabet` are stack-encoded, a DFA is induced over the encoding, and the result is decoded to a :class:`~sofic.shifts.sofic_dyck.SoficDyckShift` :cite:`Muskardin2025`: @@ -170,13 +181,13 @@ induced over the encoding, and the result is decoded to a shift = learn_sofic_dyck_shift_papni(positive=["()", "(())"], negative=["("], alphabet=alphabet) For fitting probabilities on the learned topology, see -:doc:`../generators/stack_inference`. +:doc:`../inference/stack_cssr`. -.. autoclass:: sofic.automata.papni.DyckAlphabet +.. autoclass:: sofic.automata.learning.papni.DyckAlphabet :members: classify, symbol_alphabet -.. autofunction:: sofic.automata.papni.learn_sofic_dyck_shift_papni -.. autofunction:: sofic.automata.papni.is_well_matched -.. autofunction:: sofic.automata.papni.papni_encode -.. autofunction:: sofic.automata.papni.papni_encode_samples -.. autofunction:: sofic.automata.papni.sofic_dyck_shift_from_papni_dfa +.. autofunction:: sofic.automata.learning.papni.learn_sofic_dyck_shift_papni +.. autofunction:: sofic.automata.learning.papni.is_well_matched +.. autofunction:: sofic.automata.learning.papni.encode_dyck_word +.. autofunction:: sofic.automata.learning.papni.encode_dyck_samples +.. autofunction:: sofic.automata.learning.papni.sofic_dyck_shift_from_papni_dfa diff --git a/docs/automata/nfa.rst b/docs/automata/nfa.rst index 67f6c30..64983e6 100644 --- a/docs/automata/nfa.rst +++ b/docs/automata/nfa.rst @@ -31,4 +31,4 @@ API === .. autoclass:: NFA - :members: add_transition, recognizes, union, intersection, intersect, complement, difference, concat, concatenate, kleene_star, star, determinize, minimize + :members: add_transition, recognizes, union, intersection, complement, difference, concat, kleene_star, determinize, minimize diff --git a/docs/automata/nwa.rst b/docs/automata/nwa.rst index 2fabf4e..6b060da 100644 --- a/docs/automata/nwa.rst +++ b/docs/automata/nwa.rst @@ -53,6 +53,11 @@ visible roles. Use :meth:`NestedWordAutomaton.to_vpa` to encode an NWA as a VPA; by default symbols are tagged with their role so overlapping NWA alphabets still become a disjoint visible alphabet. +The closure operations (``union``, ``intersection``, ``complement``, +``difference``, ``concat``, ``kleene_star``) and decision procedures +(``is_empty``, ``is_universal``, ``includes``, ``equivalent``) run on the tagged +VPA encoding (see :doc:`vpa`) and are translated back to an NWA. + API === @@ -60,6 +65,8 @@ API :members: from_visible_word, validate .. autoclass:: NestedWordAutomaton - :members: add_call_transition, add_return_transition, add_internal_transition, recognizes, recognizes_visible, from_vpa, to_vpa + :members: add_call_transition, add_return_transition, add_internal_transition, recognizes, recognizes_visible, + from_vpa, to_vpa, union, intersection, complement, difference, concat, kleene_star, is_empty, + is_universal, includes, equivalent .. autofunction:: sofic.automata.nwa_simulation.recognizes_nwa diff --git a/docs/automata/observation_table.rst b/docs/automata/observation_table.rst index a384074..ea2b29c 100644 --- a/docs/automata/observation_table.rst +++ b/docs/automata/observation_table.rst @@ -1,5 +1,5 @@ .. observation_table.rst -.. py:module:: sofic.automata.observation +.. py:module:: sofic.automata.learning.observation ***************** Observation Table @@ -15,6 +15,7 @@ API .. autoclass:: ObservationTable -.. autofunction:: sofic.automata.canonical_extraction.observation_to_canonical_rfsa -.. autofunction:: sofic.automata.canonical_extraction.observation_to_atomaton -.. autofunction:: sofic.automata.canonical_extraction.observation_to_minimal_dfa +.. autofunction:: sofic.automata.canonical.residual.observation_to_canonical_rfsa +.. autofunction:: sofic.automata.canonical.residual.observation_to_atomaton +.. autofunction:: sofic.automata.canonical.residual.observation_to_maximized_prime_atomaton +.. autofunction:: sofic.automata.canonical.residual.observation_to_minimal_dfa diff --git a/docs/automata/rfsa.rst b/docs/automata/rfsa.rst index ea30abd..3f23c22 100644 --- a/docs/automata/rfsa.rst +++ b/docs/automata/rfsa.rst @@ -1,19 +1,43 @@ .. rfsa.rst -.. py:module:: sofic.automata.rfsa +.. py:module:: sofic.automata.canonical.rfsa *** RFSA *** -Residual finite state automata (:class:`ResidualFiniteStateAutomaton`) and their -canonical form (:class:`CanonicalRFSA`) follow the residual-language theory of -Denis, Lemay, and Terlutte :cite:`Denis2002`. The extraction helpers documented -here expose RFSA-oriented entry points without claiming a fully minimized RFSA -pipeline beyond the implemented automata-backed construction. +Residual finite state automata (:class:`ResidualFiniteStateAutomaton`) are NFAs +whose every state accepts a residual (left quotient) of the language +:cite:`Denis2002`. :meth:`ResidualFiniteStateAutomaton.validate` checks this +exactly against the minimal DFA. + +The canonical RFSA (:class:`CanonicalRFSA`) has one state per *prime* +residual -- a non-empty residual that is not the union of the residuals strictly +inside it -- with initial states the primes contained in the language, accepting +states the primes containing the empty word, and a transition +:math:`p \xrightarrow{a} p'` whenever :math:`L_{p'} \subseteq a^{-1} L_p` +:cite:`Denis2002`. It is never larger than the minimal DFA and can be +exponentially smaller: for :math:`\Sigma^* a \Sigma^n` the minimal DFA has +:math:`2^{n+1}` states and the canonical RFSA :math:`n + 2`. + +Reversing a canonical RFSA gives the maximized prime átomaton of the reversed +language (:meth:`CanonicalRFSA.dual`; see :doc:`atomaton`). NL\* learns the +canonical RFSA from queries (:doc:`learning`). + +.. code-block:: python + + from sofic.automata.canonical.rfsa import CanonicalRFSA + + rfsa = CanonicalRFSA.from_language(nfa) + rfsa.validate() # every state accepts a residual + rfsa.dual() # maximized prime átomaton of the reverse API === .. autoclass:: ResidualFiniteStateAutomaton .. autoclass:: CanonicalRFSA - :members: from_language, from_observation_table + :members: from_language, from_observation_table, dual + +.. autofunction:: sofic.automata.canonical.residual.canonical_rfsa_from_language +.. autoclass:: sofic.automata.canonical.residual.ResidualTable + :members: includes, is_covered, prime_states diff --git a/docs/automata/subsequential.rst b/docs/automata/subsequential.rst index 870ebf7..cdb0da7 100644 --- a/docs/automata/subsequential.rst +++ b/docs/automata/subsequential.rst @@ -31,9 +31,9 @@ hierarchy :cite:`Mohri2009`. In [5]: from sofic.automata.subsequential import WeightedFiniteStateTransducer - In [6]: from sofic.examples.processes import BinaryChannel + In [6]: from sofic.examples.processes import binary_channel - In [7]: w = WeightedFiniteStateTransducer.from_transducer(BinaryChannel(0.1, 0.2)) + In [7]: w = WeightedFiniteStateTransducer.from_transducer(binary_channel(0.1, 0.2)) @doctest float In [8]: w.weight(['0'], ['0']) diff --git a/docs/automata/transducers.rst b/docs/automata/transducers.rst index c301b78..262dfa2 100644 --- a/docs/automata/transducers.rst +++ b/docs/automata/transducers.rst @@ -29,15 +29,15 @@ Composition The composition helpers mirror the common ``cmpy`` transducer operations: -* :func:`sofic.automata.transducer_operations.compose_tt` serially composes +* :func:`sofic.automata.transducer_operations.compose_transducers` serially composes transducers. -* :func:`sofic.automata.transducer_operations.compose_tg` composes a +* :func:`sofic.automata.transducer_operations.compose_transducer_generator` composes a transducer with a stochastic generator and returns a joint input/output generator. * :func:`sofic.automata.transducer_operations.transduce_generator` returns the output-only generator induced by driving a transducer with a generator. -* :func:`sofic.automata.transducer_operations.cartesian_product_tt` and - :func:`sofic.automata.transducer_operations.cartesian_product_gg` build +* :func:`sofic.automata.transducer_operations.transducer_product` and + :func:`sofic.automata.transducer_operations.generator_product` build tuple-symbol Cartesian products. For convenience, :class:`MealyMachine` also exposes ``compose``, @@ -55,8 +55,8 @@ API .. autofunction:: sofic.automata.transducer_simulation.transduce_mealy .. autofunction:: sofic.automata.transducer_simulation.transduce_moore -.. autofunction:: sofic.automata.transducer_operations.compose_tt -.. autofunction:: sofic.automata.transducer_operations.compose_tg +.. autofunction:: sofic.automata.transducer_operations.compose_transducers +.. autofunction:: sofic.automata.transducer_operations.compose_transducer_generator .. autofunction:: sofic.automata.transducer_operations.transduce_generator -.. autofunction:: sofic.automata.transducer_operations.cartesian_product_tt -.. autofunction:: sofic.automata.transducer_operations.cartesian_product_gg +.. autofunction:: sofic.automata.transducer_operations.transducer_product +.. autofunction:: sofic.automata.transducer_operations.generator_product diff --git a/docs/automata/vpa.rst b/docs/automata/vpa.rst index 35b54ce..33884e6 100644 --- a/docs/automata/vpa.rst +++ b/docs/automata/vpa.rst @@ -6,107 +6,116 @@ Visibly Pushdown Automata ************************* :class:`VisiblyPushdownAutomaton` partitions the alphabet into call, return, -and internal symbols. Call transitions push a stack symbol, return transitions -may either be guarded by a stack symbol or left unguarded as a wildcard over -ordinary stack entries, and internal transitions leave the stack untouched. -The model and its nested-word connection follow Alur and Madhusudan -:cite:`AlurMadhusudan2009`. +and internal symbols :cite:`AlurMadhusudan2009`. Call transitions push a stack +symbol, internal transitions leave the stack alone, and return transitions pop +it. A return guarded by a stack symbol fires only when that symbol is on top; a +wildcard return (no stack symbol) fires on every stack symbol, and also on the +empty stack when the VPA has a ``bottom_stack_symbol``. A return on the empty +stack is a *pending return*; it is possible only with a bottom symbol and leaves +the stack empty. Acceptance is by final state, so words may end with *pending +calls* still on the stack. + +Operations and decisions +======================== + +Every closure operation returns a concrete automaton: + +* ``union`` (disjoint sum) and ``intersection`` (synchronized product); +* ``determinize`` -- the summary construction of :cite:`AlurMadhusudan2009`, + whose states pair a summary relation with the set of current states. The + result is a complete :class:`DeterministicVisiblyPushdownAutomaton`, and + :meth:`DeterministicVisiblyPushdownAutomaton.from_vpa` uses it whenever its + input is nondeterministic; +* ``complement`` (determinize, then flip accepting states) and ``difference``; +* ``concat`` and ``kleene_star``. Each factor is read from an empty stack of its + own: the finite control records whether the current factor's stack is empty + and pushes that bit with every symbol, so a return that would pop a pending + call of an earlier factor counts as a pending return of the current one. + +Emptiness is decided by saturating the relation of well-matched summaries and +then searching states reachable with pending calls or pending returns; +``accepted_word`` returns a witness. ``is_universal``, ``includes``, +``equivalent``, and ``has_unmatched_word`` build on it. +:class:`~sofic.automata.nwa.NestedWordAutomaton` exposes the same operations by +delegating through :meth:`~sofic.automata.nwa.NestedWordAutomaton.to_vpa`. -Canonical forms -=============== +.. code-block:: python + + balanced.union(other).equivalent(other.union(balanced)) # True + balanced.complement().complement().equivalent(balanced) # True + balanced.concat(balanced).accepted_word() # e.g. ('(', ')') + +Canonical and modular forms +=========================== -The VPA module includes four deterministic canonical forms: +General visibly pushdown languages have no unique minimal deterministic VPA, and +exact unrestricted minimization is NP-complete :cite:`Gauwin2020`. Canonical +forms exist for well-matched languages, or once calls are assigned to modules +:cite:`AlurKumarMadhusudanViswanathan2005`. + +``CanonicalVisiblyPushdownAutomaton`` + The Myhill-Nerode canonical deterministic VPA of a well-matched language. + Its states are classes of the finite algebra of well-matched summaries, + refined jointly for top-level contexts and for contexts inside a pending + call. With an empty call alphabet it is the minimal DFA. Languages with a + pending call or return raise + :exc:`~sofic.exceptions.NonWellMatchedLanguageError`. ``SingleEntryVisiblyPushdownAutomaton`` A k-module SEVPA. Modules partition the states, calls are assigned to - modules by ``call_partition``, every non-base module has one entry state in - ``entry_states``, and every call pushes ``(caller_state, call_symbol)``. + modules by ``call_partition``, every non-base module has one entry state, and + every call pushes ``(caller_state, call_symbol)``. ``MultipleEntryVisiblyPushdownAutomaton`` - A k-module MEVPA. Calls are still assigned to modules, but a module may have - several entries. The pushed call stack symbol must depend only on the source - state. - -``CallDrivenAutomaton`` - A CDA, used here as the shared modular generalization. The target of a call - transition is determined by the call symbol, independent of the source state. - -``CanonicalVisiblyPushdownAutomaton`` - The Myhill-Nerode canonical deterministic VPA. It is constructed from the - finite algebra of well-matched summaries induced by a deterministic VPA and - quotiented by finite right-context acceptance signatures. If the call - alphabet is empty, this construction specializes to the usual minimal DFA - right congruence. - -The modular ``minimize`` constructors require deterministic input and fixed -module/call metadata. They intentionally do not attempt arbitrary VPA -minimization: visibly pushdown automata do not have unique minimum recognizers -in general, and exact unrestricted minimization is NP-complete. - -Operations -========== - -Finite automata expose the usual regular operations directly on ``DFA`` and -``NFA`` instances: ``union``, ``intersection``/``intersect``, ``complement``, -``difference``, ``concat``/``concatenate``, and ``kleene_star``/``star``. - -VPAs expose the same operation names. These return -``CompositeVisiblyPushdownAutomaton`` instances, which are exact VPA language -expressions with a ``recognizes`` method. This keeps concatenation and Kleene -star correct even when an operand accepts with pending stack content; concrete -graph normalization for those composite VPAs is intentionally left separate -from the operation API. - -Constructor sketch -================== + A k-module MEVPA. A module may have several entries, and the pushed symbol + depends only on the caller state. + +``ModularVisiblyPushdownAutomaton`` + The shared modular generalization: a call's target depends only on the call + symbol. + +:func:`~sofic.automata.vpa.operations.to_single_entry` and +:func:`~sofic.automata.vpa.operations.to_multiple_entry` convert any VPA of a +well-matched language into these forms (default: one module per call symbol). +On a call the state resets to the module's entry and the caller is pushed, so +the automaton forgets its caller; that is why pending calls -- and hence +non-well-matched languages, which raise +:exc:`~sofic.exceptions.NonWellMatchedLanguageError` -- are out of scope. The +modular ``minimize`` constructors call these conversions when no modules are +given, and otherwise quotient the supplied module structure. .. code-block:: python + SingleEntryVisiblyPushdownAutomaton.minimize(vpa) # convert, then minimize SingleEntryVisiblyPushdownAutomaton.minimize( - vpa, - call_partition={"call": "module"}, - modules={"main": {"q0"}, "module": {"entry", "body"}}, - entry_states={"module": "entry"}, + sevpa, call_partition={"call": "module"}, modules=sevpa.modules, ) - - MultipleEntryVisiblyPushdownAutomaton.minimize( - vpa, - modules={"main": {"q0"}, "module": {"entry0", "entry1"}}, - call_partition={"call0": "module", "call1": "module"}, - ) - - CallDrivenAutomaton.minimize( - vpa, - modules={"main": {"q0"}, "module": {"entry"}}, - call_partition={"call": "module"}, - ) - CanonicalVisiblyPushdownAutomaton.from_vpa(vpa) -References -========== - -The summary and SEVPA constructions follow Alur, Kumar, Madhusudan, and -Viswanathan :cite:`AlurKumarMadhusudanViswanathan2005`. The unrestricted -minimization limitation follows Gauwin, Muscholl, and Raskin -:cite:`Gauwin2020`. - API === .. autoclass:: VisiblyPushdownAutomaton - :members: union, intersection, intersect, complement, difference, concat, concatenate, kleene_star, star + :members: union, intersection, complement, difference, concat, kleene_star, determinize, is_empty, + accepted_word, is_universal, includes, equivalent, has_unmatched_word .. autoclass:: DeterministicVisiblyPushdownAutomaton - -.. autoclass:: CompositeVisiblyPushdownAutomaton + :members: from_vpa .. autoclass:: SingleEntryVisiblyPushdownAutomaton + :members: minimize .. autoclass:: MultipleEntryVisiblyPushdownAutomaton + :members: minimize -.. autoclass:: CallDrivenAutomaton +.. autoclass:: ModularVisiblyPushdownAutomaton + :members: minimize .. autoclass:: CanonicalVisiblyPushdownAutomaton + :members: from_vpa + +.. automodule:: sofic.automata.vpa.operations + :members: normalize, determinize, complement, concat, kleene_star, well_matched_summaries, accepted_word, + has_unmatched_word, to_single_entry, to_multiple_entry -.. autofunction:: sofic.automata.vpa_simulation.recognizes_vpa +.. autofunction:: sofic.automata.vpa.simulation.recognizes_vpa diff --git a/docs/automata/wheeler.rst b/docs/automata/wheeler.rst index 5274d3f..c347bc9 100644 --- a/docs/automata/wheeler.rst +++ b/docs/automata/wheeler.rst @@ -71,7 +71,7 @@ What the order buys uses this to replace ``2^n`` subsets with ``n(n+1)/2`` intervals, which makes the Markov order, cryptic order, and reset threshold polynomial. * Determinization to at most ``2n - 1 - |Sigma|`` states - (:func:`wnfa_to_wdfa`) and a unique minimal WDFA (:func:`minimum_wdfa`), + (:func:`determinize_wheeler`) and a unique minimal WDFA (:func:`minimize_wheeler`), both of which fail for general automata. * A succinct index — see below. @@ -100,8 +100,8 @@ Width Canonical forms and minimization ================================ -.. autofunction:: minimum_wdfa -.. autofunction:: wnfa_to_wdfa +.. autofunction:: minimize_wheeler +.. autofunction:: determinize_wheeler .. autofunction:: wheeler_canonical_form .. autofunction:: wheeler_isomorphic .. autofunction:: wheeler_state_index diff --git a/docs/core/serialization.rst b/docs/core/serialization.rst index bcfc090..6cf794a 100644 --- a/docs/core/serialization.rst +++ b/docs/core/serialization.rst @@ -26,7 +26,7 @@ The instance methods :meth:`~sofic.core.StateMachine.to_yaml`, :meth:`~sofic.core.StateMachine.write_yaml`, :meth:`~sofic.core.StateMachine.from_yaml`, and :meth:`~sofic.core.StateMachine.read_yaml` delegate to the module-level -functions below. The polymorphic :func:`model_from_yaml` / :func:`from_yaml` +functions below. The polymorphic :func:`model_from_yaml` / :func:`read_yaml` readers reconstruct the correct subclass from the serialized class tag, so they are convenient when the concrete type is not known in advance. @@ -35,7 +35,6 @@ API .. autofunction:: model_to_yaml .. autofunction:: model_from_yaml -.. autofunction:: from_yaml .. autofunction:: read_yaml .. autofunction:: model_to_dict .. autofunction:: model_from_dict diff --git a/docs/examples.rst b/docs/examples.rst index 566f186..071a7bd 100644 --- a/docs/examples.rst +++ b/docs/examples.rst @@ -165,22 +165,43 @@ Process library --------------- In addition to the curated ε-machines above, :mod:`sofic.examples.processes` -ports a large library of parametrized process factories (``GoldenMean``, -``Even``, ``Nemo``, ``IID``, ``Ising``, ``Ehrenfest``, the periodic and -``Misiurewicz`` families, and many more). Each is a function that returns a -generator, defaulting to an :class:`~sofic.generators.epsilon_machine.EpsilonMachine` -but accepting a ``machine_type`` argument: +ports a large library of parametrized process factories from cmpy +(``golden_mean_forbid_00``, ``iid``, ``ising``, ``ehrenfest``, the periodic and +``misiurewicz`` families, and many more). Every factory name is snake_case. Each +is a function that returns a generator, defaulting to an +:class:`~sofic.generators.epsilon_machine.EpsilonMachine` but accepting a +``machine_type`` argument: .. ipython:: - In [1]: from sofic.examples import GoldenMean, Even, Nemo + In [1]: from sofic.examples import golden_mean_forbid_00 - In [2]: gm = GoldenMean(bias=0.5) + In [2]: gm = golden_mean_forbid_00(bias=0.5) @doctest float In [3]: gm.entropy_rate() Out[3]: 0.6666666666666665 +cmpy factories that merely duplicated a curated example are not ported; use the +curated function instead: cmpy's ``BiasedCoin(b)`` is :func:`bernoulli` ``(b)``, +``FairCoin`` is :func:`fair_coin`, ``Even(bias=b)`` is :func:`even_process` +``(b)``, ``Nemo(p=p, q=q)`` is :func:`nemo_process` ``(p, q)``, ``NRPS`` is +:func:`noisy_random_phase_slip`, and ``ABC(p, q)`` is +:func:`alternating_biased_coins` ``(1 - p, 1 - q)``. The curated versions emit +integer symbols ``0``/``1`` (except :func:`bernoulli`), whereas the ported +factories emit strings ``"0"``/``"1"``. + +Several remaining factories share an implementation with a curated example and +differ only in string symbols, state names, or parametrization: +``golden_mean_forbid_00(b)`` is :func:`golden_mean_forward` ``(1 - b)`` (and +:func:`golden_mean_markov` ``(b)`` with ``A``/``B`` swapped), ``restricted_gm`` +is :func:`restricted_golden_mean`, and ``irreversible_two_state()`` is +:func:`ellison_fig9_forward`. Similar names do not always mean the same process: +:func:`golden_mean` (forbids ``11``) is the ``0 <-> 1`` mirror of +``golden_mean_forbid_00``, and ``butterfly_two_branch`` and +``phase_slip_backtrack_cmpy`` differ from :func:`butterfly_process` and +:func:`~sofic.examples.epsilon_machines.phase_slip_backtrack`. + The module also exposes registries — ``processes.process_list`` and ``processes.transducer_list`` — that enumerate every factory, which is handy for parametrized tests and sweeps: @@ -192,8 +213,8 @@ parametrized tests and sweeps: In [5]: len(processes.process_list) > 0 Out[5]: True -A parallel set of transducer factories (``BitFlip``, ``Parity``, ``Delay``, -``BinaryChannel``, …) lives alongside the processes and produces +A parallel set of transducer factories (``bit_flip``, ``parity``, ``delay``, +``binary_channel``, …) lives alongside the processes and produces :class:`~sofic.automata.transducers.MealyMachine` instances. Symbolic-shift examples (:mod:`sofic.examples.shifts`) provide the diff --git a/docs/generators/epsilon_machine.rst b/docs/generators/epsilon_machine.rst index dc7d3af..e6e3264 100644 --- a/docs/generators/epsilon_machine.rst +++ b/docs/generators/epsilon_machine.rst @@ -87,7 +87,7 @@ structure can differ, because a cycle weight may equal one only by algebraic coincidence in the probabilities. See also :doc:`bidirectional_epsilon_machine`, :doc:`information_anatomy`, -:doc:`block_convergence`, and :doc:`epsilon_inference` (sample-based reconstruction). +:doc:`block_convergence`, and :doc:`../inference/cssr` (sample-based reconstruction). API === diff --git a/docs/generators/epsilon_transducer.rst b/docs/generators/epsilon_transducer.rst index 3ff0ffe..d53ab53 100644 --- a/docs/generators/epsilon_transducer.rst +++ b/docs/generators/epsilon_transducer.rst @@ -26,22 +26,22 @@ Minimize a joint-unifilar stochastic transducer to its causal states: In [1]: from sofic import EpsilonTransducer - In [2]: from sofic.examples.processes import BinaryChannel, GMtoEven + In [2]: from sofic.examples.processes import binary_channel, gm_to_even - In [3]: eps = EpsilonTransducer.from_channel(BinaryChannel(0.1, 0.2)) + In [3]: eps = EpsilonTransducer.from_channel(binary_channel(0.1, 0.2)) @doctest In [4]: len(list(eps.states())) Out[4]: 1 @doctest - In [5]: EpsilonTransducer.from_channel(GMtoEven()).is_unifilar() + In [5]: EpsilonTransducer.from_channel(gm_to_even()).is_unifilar() Out[5]: True The channel can also be read off a joint ``(input, output)`` generator with :meth:`~EpsilonTransducer.from_joint_generator`, or reconstructed from paired sample sequences with :meth:`~EpsilonTransducer.from_paired_sequences` (the -transCSSR algorithm, :doc:`epsilon_transducer_inference`). +transCSSR algorithm, :doc:`../inference/transcssr`). Channel measures ================ diff --git a/docs/generators/generators.rst b/docs/generators/generators.rst index 7f66ba2..87b695b 100644 --- a/docs/generators/generators.rst +++ b/docs/generators/generators.rst @@ -50,17 +50,6 @@ Constructions and conversions conversions lumping -Inference -========= - -.. toctree:: - :maxdepth: 1 - - hmm_inference - epsilon_inference - epsilon_transducer_inference - stack_inference - Advanced ======== diff --git a/docs/generators/information_anatomy.rst b/docs/generators/information_anatomy.rst index 2732bd8..c04382e 100644 --- a/docs/generators/information_anatomy.rst +++ b/docs/generators/information_anatomy.rst @@ -120,7 +120,7 @@ forward and reverse structural ephemeral branches match. .. ipython:: - In [1]: from sofic.examples import nemo_process, NRPS + In [1]: from sofic.examples import nemo_process, noisy_random_phase_slip In [2]: nemo = nemo_process().to_bidirectional() @@ -128,7 +128,7 @@ forward and reverse structural ephemeral branches match. In [3]: round(nemo.internal_markov_entropy_rate(), 6), round(nemo.reverse_internal_markov_entropy_rate(), 6) Out[3]: (0.5, 0.5) - In [4]: nrps = NRPS().to_bidirectional() + In [4]: nrps = noisy_random_phase_slip().to_bidirectional() # An arrow of time (r_fwd != r_rev): forward and reverse chain rates differ. In [5]: round(nrps.internal_markov_entropy_rate(), 6), round(nrps.reverse_internal_markov_entropy_rate(), 6) diff --git a/docs/generators/quasi_realization.rst b/docs/generators/quasi_realization.rst index 376c9ec..fb6928a 100644 --- a/docs/generators/quasi_realization.rst +++ b/docs/generators/quasi_realization.rst @@ -16,6 +16,6 @@ API .. autoclass:: QuasiRealization .. autoclass:: QuasiStochasticModel -.. autofunction:: sofic.generators.quasi_inference.transition_matrices +.. autofunction:: sofic.generators.quasi_inference.symbol_matrices .. autofunction:: sofic.generators.quasi_inference.stationary_quasidistribution .. autofunction:: sofic.generators.quasi_inference.word_probability diff --git a/docs/generators/epsilon_inference.rst b/docs/inference/cssr.rst similarity index 76% rename from docs/generators/epsilon_inference.rst rename to docs/inference/cssr.rst index a9494fb..77cb6b7 100644 --- a/docs/generators/epsilon_inference.rst +++ b/docs/inference/cssr.rst @@ -1,5 +1,5 @@ -.. epsilon_inference.rst -.. py:module:: sofic.generators.epsilon_inference +.. cssr.rst +.. py:module:: sofic.inference.cssr ************************** ε-Machine inference @@ -29,7 +29,7 @@ Quick start observations, _ = oracle.sample(5000, rng) inferred = EpsilonMachine.from_sequence( - observations, method="cssr", Lmax=4, alpha=0.001 + observations, method="cssr", max_history=4, alpha=0.001 ) len(list(inferred.states())) # 2 for the even process @@ -39,7 +39,7 @@ CSSR CSSR :cite:`Shalizi2002` starts from an IID model and grows causal states in three phases: 1. **Initialize** — one state for the empty history. -2. **Homogenize** — extend each suffix one symbol into the past, up to ``Lmax``; a +2. **Homogenize** — extend each suffix one symbol into the past, up to ``max_history``; a child suffix whose next-symbol distribution differs significantly from its state's (G-test, :math:`\chi^2`, Monte Carlo exact G-test, or total-variation threshold) moves to the best @@ -47,23 +47,23 @@ CSSR :cite:`Shalizi2002` starts from an IID model and grows causal states in thr 3. **Determinize** — drop transient states, then split states until each state and symbol lead to a single successor, then keep the most-visited recurrent class. -A length-``Lmax`` suffix has no one-symbol extension in the suffix tree, so its +A length-``max_history`` suffix has no one-symbol extension in the suffix tree, so its successor drops the oldest symbol. For a non-Markovian process that can forget the -phase: in the even process with ``Lmax = 3``, the successor of ``011`` on ``1`` would -be the ambiguous ``111``. So the length-``Lmax + 1`` suffix (here ``0111``) is tested +phase: in the even process with ``max_history = 3``, the successor of ``011`` on ``1`` would +be the ambiguous ``111``. So the length-``max_history + 1`` suffix (here ``0111``) is tested against the truncated suffix's state, and is sent to the best matching state when the two differ. -Choose ``Lmax`` at least the synchronization length of the source (its order, for +Choose ``max_history`` at least the synchronization length of the source (its order, for a Markov source). Much larger values run many more significance tests, and some split states by chance; lowering ``alpha`` counters this. A process that is not -exactly synchronizable has no finite-``Lmax`` reconstruction, and CSSR returns +exactly synchronizable has no finite-``max_history`` reconstruction, and CSSR returns extra states. -Choosing ``Lmax`` and calibrating the tests -------------------------------------------- +Choosing ``max_history`` and calibrating the tests +-------------------------------------------------- -``Lmax="auto"`` sets ``Lmax`` to :func:`suggest_lmax`, the Markov order estimated +``max_history="auto"`` sets ``max_history`` to :func:`suggest_max_history`, the Markov order estimated by :func:`dit.inference.select_markov_order`. Its default method tests order :math:`n` against :math:`n + 1` with surrogates that preserve the observed :math:`(n + 1)`-gram counts exactly, so the test holds its nominal size at any @@ -76,7 +76,11 @@ it as the longest history the data supports, not as a synchronization length. The morph tests also rely on the chi-squared limit, which fails for the sparse counts of long suffixes. ``test="exact"`` compares the G statistic with tables drawn uniformly given the observed margins whenever an expected count is below 5 -(seeded from the table, so reconstruction stays deterministic). +(seeded from the table, so reconstruction stays deterministic). The G-test +(``test="g"``) applies Yates' continuity correction when the table has one degree +of freedom (two observed symbols), matching +``scipy.stats.chi2_contingency(table, lambda_="log-likelihood")``; transCSSR and +stack CSSR share the same implementation. ``correction="bonferroni"`` divides ``alpha`` by the number of suffixes eligible for testing, bounding the chance of any spurious split. Because CSSR chooses each test in light of earlier outcomes, false-discovery-rate step-up procedures do not @@ -85,17 +89,17 @@ apply directly. .. code-block:: python inferred = EpsilonMachine.from_sequence( - observations, method="cssr", Lmax="auto", test="exact", correction="bonferroni" + observations, method="cssr", max_history="auto", test="exact", correction="bonferroni" ) -.. autofunction:: cssr +.. autofunction:: learn_epsilon_machine_cssr -.. autofunction:: suggest_lmax +.. autofunction:: suggest_max_history After reconstruction, check the result with :func:`~sofic.inference.diagnostics.goodness_of_fit` and :func:`~sofic.inference.diagnostics.structure_stability` -(see :doc:`../inference/diagnostics`). +(see :doc:`diagnostics`). .. autofunction:: morphs_differ @@ -108,11 +112,11 @@ to a unifilar presentation. With ``delta=0``, two morphs are equivalent unless G-test at significance 0.01 tells them apart, a tolerance that scales with the sample. Transitions follow the same successor rule as CSSR. -``subtree_merge`` accepts ``alpha``, ``test`` (including ``"exact"``) and +``learn_epsilon_machine_subtree`` accepts ``alpha``, ``test`` (including ``"exact"``) and ``correction="bonferroni"``, which divides ``alpha`` over the history pairs -compared, as well as ``L="auto"``. +compared, as well as ``max_history="auto"``. -.. autofunction:: subtree_merge +.. autofunction:: learn_epsilon_machine_subtree Spectral reconstruction ======================= @@ -135,22 +139,22 @@ process). Otherwise the Hankel singular-value gap selects the rank. observations, method="spectral", prefix_length=3, rank=2 ) -.. autofunction:: spectral +.. autofunction:: sofic.inference.spectral.learn_epsilon_machine_spectral Unified entry point =================== Use :meth:`~sofic.generators.epsilon_machine.EpsilonMachine.from_sequence` to dispatch to CSSR, subtree merging, or spectral reconstruction -(see :doc:`epsilon_machine`). +(see :doc:`../generators/epsilon_machine`). Related inference methods ========================= -**transCSSR** — input/output ε-transducers; see :doc:`epsilon_transducer_inference`. +**transCSSR** — input/output ε-transducers; see :doc:`transcssr`. **Bayesian structural inference** — conjugate Dirichlet–multinomial evidence over -candidate unifilar topologies :cite:`Strelioff2014`; see :doc:`../inference/epsilon`. +candidate unifilar topologies :cite:`Strelioff2014`; see :doc:`epsilon`. This is not a Gibbs clustering heuristic over history labels. Subtree merging is the literature reconstruction by morph clustering; a separate @@ -169,5 +173,5 @@ the causal-state construction used after spectral learning. **RKHS ε-machines** — continuous-time extension (arXiv:2011.14821). -See also :doc:`epsilon_machine`, :doc:`constructions`, :doc:`hmm_inference`, -and :doc:`../inference/spectral`. +See also :doc:`../generators/epsilon_machine`, :doc:`../generators/constructions`, :doc:`hmm`, +and :doc:`spectral`. diff --git a/docs/inference/diagnostics.rst b/docs/inference/diagnostics.rst index a674dc9..7fa76f5 100644 --- a/docs/inference/diagnostics.rst +++ b/docs/inference/diagnostics.rst @@ -15,7 +15,7 @@ Goodness of fit :func:`goodness_of_fit` is a parametric bootstrap :cite:`Efron1993`. It simulates sequences as long as the data from the fitted machine and compares a -length-``L`` word statistic of the data with its distribution over the +length-``block_length`` word statistic of the data with its distribution over the simulations. Two statistics are available: * ``"g"`` — the G statistic of the observed word counts against the machine's @@ -25,17 +25,17 @@ simulations. Two statistics are available: Because the null distribution is simulated, overlapping windows need no correction. A small p-value means the machine misses structure. For CSSR that -usually means ``Lmax`` is shorter than the source's synchronization length. +usually means ``max_history`` is shorter than the source's synchronization length. .. code-block:: python - from sofic.generators.epsilon_inference import cssr + from sofic.inference.cssr import learn_epsilon_machine_cssr from sofic.inference.diagnostics import goodness_of_fit - machine = cssr(data, Lmax=1) - goodness_of_fit(machine, data, L=6).pvalue # small for the even process - machine = cssr(data, Lmax=4) - goodness_of_fit(machine, data, L=6).pvalue # large + machine = learn_epsilon_machine_cssr(data, max_history=1) + goodness_of_fit(machine, data, block_length=6).pvalue # small for the even process + machine = learn_epsilon_machine_cssr(data, max_history=4) + goodness_of_fit(machine, data, block_length=6).pvalue # large Observed words the machine forbids are listed in ``forbidden_words``. @@ -51,7 +51,7 @@ words the source never emits and can add spurious states. For example, on even-process data it returns 6–12-state machines where subsampling returns the true 2 states. -:func:`reconstruction_sweep` reconstructs over a grid of ``alpha`` and ``Lmax``. +:func:`reconstruction_sweep` reconstructs over a grid of ``alpha`` and ``max_history``. A structure that persists over a range of settings is better supported than one that appears at a single setting. diff --git a/docs/inference/hdp_hmm.rst b/docs/inference/hdp_hmm.rst index 6a6ae46..400083b 100644 --- a/docs/inference/hdp_hmm.rst +++ b/docs/inference/hdp_hmm.rst @@ -60,7 +60,7 @@ The HDP-HMM infers the state count nonparametrically, whereas :doc:`model_selection` scores a *fixed* set of candidate orders with information criteria and :mod:`sofic.inference.bayesian` compares fixed unifilar topologies by exact Dirichlet-multinomial evidence. For point-estimate reconstruction see -CSSR in :doc:`../generators/epsilon_inference`. +CSSR in :doc:`cssr`. API === diff --git a/docs/generators/hmm_inference.rst b/docs/inference/hmm.rst similarity index 61% rename from docs/generators/hmm_inference.rst rename to docs/inference/hmm.rst index c430f29..d51819c 100644 --- a/docs/generators/hmm_inference.rst +++ b/docs/inference/hmm.rst @@ -1,5 +1,5 @@ -.. hmm_inference.rst -.. py:module:: sofic.generators.hmm_inference +.. hmm.rst +.. py:module:: sofic.inference.hmm ************* HMM Inference @@ -19,7 +19,7 @@ via the Fisher and Louis :cite:`Louis1982` identities. In [3]: obs = [0, 1, 0, 0, 1] - In [4]: from sofic.generators.hmm_inference import forward, viterbi, sample + In [4]: from sofic.generators.sampling import sample; from sofic.inference.hmm import forward, viterbi In [5]: alpha = forward(eps, obs) @@ -33,7 +33,7 @@ pairs. .. ipython:: - In [8]: from sofic.generators.hmm_inference import smooth, two_slice_marginals + In [8]: from sofic.inference.hmm import smooth, two_slice_marginals In [9]: gamma = smooth(eps, obs) # gamma[t, s] = P(X_t = s | Y) @@ -44,7 +44,7 @@ transition-graph topology fixed: .. ipython:: - In [11]: from sofic.generators.hmm_inference import baum_welch + In [11]: from sofic.inference.hmm import baum_welch In [12]: data, _states = sample(golden_mean(0.3), n=2000) @@ -63,7 +63,7 @@ parameter uncertainty at the current parameters: .. ipython:: - In [14]: from sofic.generators.hmm_inference import score, standard_errors + In [14]: from sofic.inference.hmm import score, standard_errors In [15]: g = score(eps, obs) @@ -79,7 +79,7 @@ Filtering, decoding, and sampling .. autofunction:: backward .. autofunction:: log_likelihood .. autofunction:: viterbi -.. autofunction:: sample +.. autofunction:: sofic.generators.sampling.sample Smoothing --------- @@ -99,3 +99,27 @@ Score and observed information .. autofunction:: observed_information .. autofunction:: free_parameter_labels .. autofunction:: standard_errors + +Transition matrices and start policies +-------------------------------------- + +.. py:module:: sofic.generators.matrices + :no-index: + +Every routine above works with the symbol-labeled joint transition matrices +:math:`T^{(x)}_{ij} = P(S_{t+1} = j, X_t = x \mid S_t = i)` of the model's +Mealy presentation :cite:`Rabiner1989,Ellison2009`, built in one place by +:mod:`sofic.generators.matrices`. The initial state law follows one of two +policies: + +- ``"model"`` (likelihoods, decoding, sampling, word probabilities): the + model's ``initial_distribution``, or its stationary distribution when none is + given. +- ``"stationary"`` (block and window statistics such as + ``joint_block_distribution`` and directional flow): the stationary process + law, preferring the limit reached from ``initial_distribution`` on reducible + chains. + +.. autofunction:: sofic.generators.matrices.start_vector +.. autofunction:: sofic.generators.matrices.emission_tensors +.. autofunction:: sofic.generators.matrices.symbol_matrices diff --git a/docs/inference/inference.rst b/docs/inference/inference.rst index e91692c..b0105a0 100644 --- a/docs/inference/inference.rst +++ b/docs/inference/inference.rst @@ -33,13 +33,18 @@ The historical names ``InferMC`` and ``InferEM`` are retained as aliases for For *non-Bayesian* reconstruction — Causal-State Splitting Reconstruction (CSSR), subtree merging, and spectral mixed-state extraction — see the point-estimate routines in - :doc:`../generators/epsilon_inference`, - :doc:`../generators/hmm_inference`, and - :doc:`../generators/stack_inference`. + :doc:`cssr`, + :doc:`transcssr`, + :doc:`hmm`, and + :doc:`stack_cssr`. .. toctree:: :maxdepth: 1 + hmm + cssr + transcssr + stack_cssr markov epsilon spectral diff --git a/docs/inference/model_selection.rst b/docs/inference/model_selection.rst index 6e66dd0..1271926 100644 --- a/docs/inference/model_selection.rst +++ b/docs/inference/model_selection.rst @@ -8,9 +8,11 @@ Model Selection Classical model-selection criteria for choosing the order (state count) of a fitted generator when a fully-Bayesian evidence is unavailable or undesirable. The module scores any fitted :class:`~sofic.generators.base.HiddenMarkovModel` -using the natural-log likelihood from -:func:`sofic.generators.hmm_inference.log_likelihood` and a free-parameter count -read off the transition graph: +using the log-likelihood (in bits) from +:func:`sofic.inference.hmm.log_likelihood` and a free-parameter count +read off the transition graph. Log-likelihoods, log scores, and the MDL code +length are in bits; AIC, AICc, BIC, and WAIC are computed from the natural +log-likelihood so they keep their standard deviance scale: * **AIC** :cite:`Akaike1974` and the small-sample-corrected **AICc** :cite:`HurvichTsai1989`, diff --git a/docs/inference/spectral.rst b/docs/inference/spectral.rst index 478542f..d24f959 100644 --- a/docs/inference/spectral.rst +++ b/docs/inference/spectral.rst @@ -12,7 +12,7 @@ matrix** of block probabilities, takes its singular value decomposition, and reads the observable operators off the truncated factorization :cite:`Balle2014`. It is the automata-theoretic twin of the spectral hidden Markov model algorithm of Hsu, Kakade & Zhang :cite:`Hsu2012`. Unlike -Baum-Welch (:func:`sofic.generators.hmm_inference.baum_welch`), spectral +Baum-Welch (:func:`sofic.inference.hmm.baum_welch`), spectral learning is a consistent, one-shot estimator with no local optima, and the model order is read from the singular-value spectrum instead of being fixed in advance. @@ -68,16 +68,16 @@ Projection to an ε-machine non-negative Mealy projection when one exists in the learned basis, otherwise mixed-state enumeration of the observable operators :cite:`Ellison2009`. The same path is -:func:`~sofic.generators.epsilon_inference.spectral` / +:func:`~sofic.inference.spectral.learn_epsilon_machine_spectral` / ``EpsilonMachine.from_sequence(..., method="spectral")``. .. code-block:: python - from sofic.generators.epsilon_inference import spectral + from sofic.inference.spectral import learn_epsilon_machine_spectral from sofic.examples import golden_mean process = golden_mean(0.5) - eps = spectral(word_probability=process.word_probability, alphabet=(0, 1), prefix_length=3, rank=2) + eps = learn_epsilon_machine_spectral(word_probability=process.word_probability, alphabet=(0, 1), prefix_length=3, rank=2) len(list(eps.states())) # 2 API diff --git a/docs/generators/stack_inference.rst b/docs/inference/stack_cssr.rst similarity index 69% rename from docs/generators/stack_inference.rst rename to docs/inference/stack_cssr.rst index af85695..ef1fa1c 100644 --- a/docs/generators/stack_inference.rst +++ b/docs/inference/stack_cssr.rst @@ -1,5 +1,5 @@ -.. stack_inference.rst -.. py:module:: sofic.generators.stack_inference +.. stack_cssr.rst +.. py:module:: sofic.inference.cssr.stack ********************* Stack-HMM Inference @@ -7,17 +7,17 @@ Stack-HMM Inference Inference routines that reconstruct a :class:`~sofic.generators.stack_hmm.HiddenMarkovStackModel` from sequences -over a :class:`~sofic.automata.papni.DyckAlphabet`. These extend the -finite-state inference of :doc:`epsilon_inference` with visibly pushdown stack +over a :class:`~sofic.automata.learning.papni.DyckAlphabet`. These extend the +finite-state inference of :doc:`cssr` with visibly pushdown stack semantics :cite:`BealBlockeletDima2015`. Two families are provided: * **Topology known.** Given a :class:`~sofic.shifts.sofic_dyck.SoficDyckShift` presentation, - :func:`fit_stack_hmm_mle` estimates smoothed maximum-likelihood transition + :func:`learn_stack_hmm_mle` estimates smoothed maximum-likelihood transition weights from a sample. -* **Topology unknown.** :func:`stack_cssr` and :func:`stack_subtree_merge` +* **Topology unknown.** :func:`learn_stack_hmm_cssr` and :func:`learn_stack_hmm_subtree` reconstruct both the control graph and its probabilities from a single long sequence by splitting stack-aware histories, in the spirit of Causal-State Splitting Reconstruction :cite:`Shalizi2004`. @@ -28,14 +28,14 @@ Two families are provided: .. code-block:: python from sofic.automata import DyckAlphabet - from sofic.generators import stack_cssr + from sofic.inference.cssr import learn_stack_hmm_cssr alphabet = DyckAlphabet( call_alphabet=frozenset({"("}), return_alphabet=frozenset({")"}), internal_alphabet=frozenset({"a"}), ) - model = stack_cssr(sequence, alphabet=alphabet, Lmax=4, max_stack_depth=8) + model = learn_stack_hmm_cssr(sequence, alphabet=alphabet, max_history=4, max_stack_depth=8) model.validate() Histories are counted with a bounded stack depth, so ``max_stack_depth`` caps @@ -48,18 +48,18 @@ can follow is decided by the stack top through matched call-return pairs, not by the finite control. Return edges are matched only to calls observed to close them. -``stack_cssr`` accepts the same calibration options as -:func:`~sofic.generators.epsilon_inference.cssr`: ``test="exact"``, -``correction="bonferroni"`` (over eligible configurations), and ``Lmax="auto"``. +``learn_stack_hmm_cssr`` accepts the same calibration options as +:func:`~sofic.inference.cssr.learn_epsilon_machine_cssr`: ``test="exact"``, +``correction="bonferroni"`` (over eligible configurations), and ``max_history="auto"``. Stack processes generally have infinite Markov order, so the automatic depth is a lower bound on the suffix length the data support. API === -.. autofunction:: stack_cssr -.. autofunction:: stack_subtree_merge -.. autofunction:: fit_stack_hmm_mle +.. autofunction:: learn_stack_hmm_cssr +.. autofunction:: learn_stack_hmm_subtree +.. autofunction:: learn_stack_hmm_mle .. autofunction:: learn_stack_hmm_papni .. autoclass:: StackSuffixCounts diff --git a/docs/generators/epsilon_transducer_inference.rst b/docs/inference/transcssr.rst similarity index 69% rename from docs/generators/epsilon_transducer_inference.rst rename to docs/inference/transcssr.rst index 707be51..6de557e 100644 --- a/docs/generators/epsilon_transducer_inference.rst +++ b/docs/inference/transcssr.rst @@ -1,11 +1,11 @@ -.. epsilon_transducer_inference.rst -.. py:module:: sofic.generators.epsilon_transducer_inference +.. transcssr.rst +.. py:module:: sofic.inference.cssr.transducer ********************* transCSSR Inference ********************* -transCSSR reconstructs an :doc:`ε-transducer ` from paired +transCSSR reconstructs an :doc:`ε-transducer <../generators/epsilon_transducer>` from paired input/output sample sequences, generalizing Causal-State Splitting Reconstruction :cite:`Shalizi2004` from a single process to an input/output channel :cite:`Barnett2015`. @@ -13,11 +13,13 @@ channel :cite:`Barnett2015`. Causal states are equivalence classes of joint ``(input, output)`` pasts that induce the same conditional next-output law ``P(y | history, x)`` for every input symbol ``x``. Rare histories inherit their parent's state (controlled by -``min_count``); the split decision uses a G-test at significance ``alpha``. -As in :func:`~sofic.generators.epsilon_inference.cssr`, ``test="exact"`` uses a +``min_count``); the split decision uses a G-test at significance ``alpha``, the +same test as :func:`~sofic.inference.cssr.morphs_differ` (with +Yates' continuity correction at one degree of freedom). +As in :func:`~sofic.inference.cssr.learn_epsilon_machine_cssr`, ``test="exact"`` uses a Monte Carlo exact G-test for tables with small expected counts, and ``correction="bonferroni"`` divides ``alpha`` by the number of -(history, input symbol) tests. ``Lmax="auto"`` sets the depth from the Markov +(history, input symbol) tests. ``max_history="auto"`` sets the depth from the Markov order of the joint ``(input, output)`` sequence :cite:`Pethel2014`. .. ipython:: @@ -26,15 +28,15 @@ order of the joint ``(input, output)`` sequence :cite:`Pethel2014`. In [2]: from sofic import EpsilonTransducer, MealyHMM - In [3]: from sofic.examples.processes import Delay + In [3]: from sofic.examples.processes import delay - In [4]: from sofic.automata.transducer_operations import compose_tg + In [4]: from sofic.automata.transducer_operations import compose_transducer_generator In [5]: inp = MealyHMM(observation_alphabet=frozenset({'0', '1'}), initial_distribution={'S': 1.0}) In [6]: inp.graph.add_state('S'); _ = inp.add_transition('S', 'S', '0', 0.5); _ = inp.add_transition('S', 'S', '1', 0.5); inp.validate() - In [7]: joint = compose_tg(Delay(1), inp, joint=True) + In [7]: joint = compose_transducer_generator(delay(1), inp, joint=True) In [8]: obs, _ = joint.sample(8000, np.random.default_rng(0)) @@ -47,7 +49,7 @@ order of the joint ``(input, output)`` sequence :cite:`Pethel2014`. API === -.. autofunction:: transcssr +.. autofunction:: learn_epsilon_transducer_cssr .. autoclass:: JointSuffixCounts :members: from_sequences, output_counts, state_morph diff --git a/docs/quickstart.rst.txt b/docs/quickstart.rst.txt index 9b3cf8f..61eed6d 100644 --- a/docs/quickstart.rst.txt +++ b/docs/quickstart.rst.txt @@ -77,7 +77,7 @@ topological entropy (the log of the golden ratio) :cite:`LindMarcus1995`: @doctest float In [4]: tmc.topological_entropy() - Out[4]: 0.4812118250596035 + Out[4]: 0.6942419136306174 The Parry measure turns the shift into the maximum-entropy ``MealyHMM`` on it (``tmc.parry_measure()``), the measure that attains this topological entropy. diff --git a/docs/references.bib b/docs/references.bib index afbdd2f..897990b 100644 --- a/docs/references.bib +++ b/docs/references.bib @@ -158,6 +158,28 @@ @article{Denis2002 doi = {10.3233/FUN-2002-51402}, } +@inproceedings{Tamm2015, + author = {Tamm, Hellis}, + title = {Generalization of the Double-Reversal Method of Finding a Canonical Residual Finite State Automaton}, + booktitle = {Descriptional Complexity of Formal Systems}, + series = {Lecture Notes in Computer Science}, + volume = {9118}, + pages = {268--279}, + publisher = {Springer}, + year = {2015}, + doi = {10.1007/978-3-319-19225-3_23}, +} + +@inproceedings{MaarandTamm2022, + author = {Maarand, Hendrik and Tamm, Hellis}, + title = {Yet Another Canonical Nondeterministic Automaton}, + booktitle = {Descriptional Complexity of Formal Systems}, + series = {Lecture Notes in Computer Science}, + publisher = {Springer}, + year = {2022}, + doi = {10.1007/978-3-031-13257-5_14}, +} + @inproceedings{BrzozowskiTamm2011, author = {Brzozowski, Janusz A. and Tamm, Hellis}, title = {Theory of {\'A}tomata}, diff --git a/docs/shifts/covers.rst b/docs/shifts/covers.rst index 5ce62bd..d1b1683 100644 --- a/docs/shifts/covers.rst +++ b/docs/shifts/covers.rst @@ -5,28 +5,50 @@ Covers ****** -Fischer and Krieger covers convert a Sofic shift into unifilar presentations -:cite:`Fischer1975,Krieger1984,LindMarcus1995`. - -* :class:`LeftFischerCover`, :class:`RightFischerCover` — implemented. -* :class:`LeftKriegerCover`, :class:`RightKriegerCover` — construction raises - :exc:`NotImplementedError`. +Fischer and Krieger covers are canonical resolving presentations of a sofic +shift :cite:`Fischer1975,Krieger1984,LindMarcus1995`. Following Lind and Marcus, +*right* means right-resolving (deterministic reading forward): + +* :class:`RightFischerCover` -- the unique minimal right-resolving presentation + of an irreducible sofic shift. Reducible shifts raise + :exc:`~sofic.exceptions.SoficValidationError`, since their minimal + right-resolving presentation need not be unique. +* :class:`RightKriegerCover` -- the future cover, with one vertex per follower + set of a left-infinite ray. Defined for every sofic shift; for an irreducible + shift the Fischer cover is its unique terminal component. +* :class:`LeftFischerCover`, :class:`LeftKriegerCover` -- the left-resolving + mirror images, built from the reversed shift. + +Both constructions are exact: the Fischer cover is the terminal component of the +follower-merged subset construction, and the Krieger cover's vertices are the +images of path relations lying on cycles of the finite relation monoid. .. ipython:: - In [1]: from sofic.examples import golden_mean_shift_parry; parry = golden_mean_shift_parry() + In [1]: from sofic.graph import ATTR_SYMBOL + + In [2]: from sofic.shifts import RightKriegerCover, SoficShift + + In [3]: even = SoficShift(symbol_alphabet=frozenset("01")) + + In [4]: for s, t, a in [("A", "A", "0"), ("A", "B", "1"), ("B", "A", "1")]: + ...: even.graph.add_transition(s, t, **{ATTR_SYMBOL: a}) - In [2]: parry.validate() + In [5]: len(list(RightKriegerCover.from_presentation(even).states())) API === .. autoclass:: LeftFischerCover - :members: from_sofic + :members: from_presentation .. autoclass:: RightFischerCover - :members: from_sofic + :members: from_presentation .. autoclass:: LeftKriegerCover + :members: from_presentation .. autoclass:: RightKriegerCover + :members: from_presentation -.. autofunction:: sofic.shifts.cover_construction.left_fischer_from_sofic -.. autofunction:: sofic.shifts.cover_construction.right_fischer_from_sofic +.. autofunction:: sofic.shifts.cover_construction.left_fischer_cover +.. autofunction:: sofic.shifts.cover_construction.right_fischer_cover +.. autofunction:: sofic.shifts.cover_construction.left_krieger_cover +.. autofunction:: sofic.shifts.cover_construction.right_krieger_cover diff --git a/docs/shifts/product_alphabet_shift.rst b/docs/shifts/product_alphabet_shift.rst new file mode 100644 index 0000000..890f8c1 --- /dev/null +++ b/docs/shifts/product_alphabet_shift.rst @@ -0,0 +1,36 @@ +.. product_alphabet_shift.rst +.. py:module:: sofic.shifts.product_alphabet_shift + +********************** +Product-Alphabet Shift +********************** + +A :class:`ProductAlphabetShift` is the topological support of a transducer: a +sofic subshift of the product shift on ``X x Y`` whose input and output projections are the +transducer's domain and range subshifts :cite:`LindMarcus1995`. It is the +symbolic-dynamics reading of a transducer, complementary to the +:doc:`sliding block code `. + +.. ipython:: + + In [1]: from sofic import ProductAlphabetShift + + In [2]: from sofic.examples.processes import gm_to_even + + In [3]: rel = ProductAlphabetShift.from_transducer(gm_to_even()) + + @doctest + In [4]: sorted(rel.output_alphabet()) + Out[4]: ['0', '1'] + + In [5]: input_shift = rel.input_shift() + +Build one with :meth:`~ProductAlphabetShift.from_transducer` and recover the coordinate +shifts with :meth:`~ProductAlphabetShift.input_shift` and +:meth:`~ProductAlphabetShift.output_shift`. + +API +=== + +.. autoclass:: ProductAlphabetShift + :members: from_transducer, input_shift, output_shift, input_alphabet, output_alphabet diff --git a/docs/shifts/shifts.rst b/docs/shifts/shifts.rst index 2f07371..03a0425 100644 --- a/docs/shifts/shifts.rst +++ b/docs/shifts/shifts.rst @@ -18,7 +18,7 @@ type, Sofic shifts, Dyck shifts, topological Markov chains, and covers markov_dyck_shift topological_markov_chain sliding_block_code - sofic_relation + product_alphabet_shift textile dyck_enumeration covers diff --git a/docs/shifts/sofic_relation.rst b/docs/shifts/sofic_relation.rst deleted file mode 100644 index 852c389..0000000 --- a/docs/shifts/sofic_relation.rst +++ /dev/null @@ -1,36 +0,0 @@ -.. sofic_relation.rst -.. py:module:: sofic.shifts.sofic_relation - -************** -Sofic Relation -************** - -A :class:`SoficRelation` is the topological support of a transducer: a subshift -of the product shift on ``X x Y`` whose input and output projections are the -transducer's domain and range subshifts :cite:`LindMarcus1995`. It is the -symbolic-dynamics reading of a transducer, complementary to the -:doc:`sliding block code `. - -.. ipython:: - - In [1]: from sofic import SoficRelation - - In [2]: from sofic.examples.processes import GMtoEven - - In [3]: rel = SoficRelation.from_transducer(GMtoEven()) - - @doctest - In [4]: sorted(rel.output_alphabet()) - Out[4]: ['0', '1'] - - In [5]: input_shift = rel.input_shift() - -Build one with :meth:`~SoficRelation.from_transducer` and recover the coordinate -shifts with :meth:`~SoficRelation.input_shift` and -:meth:`~SoficRelation.output_shift`. - -API -=== - -.. autoclass:: SoficRelation - :members: from_transducer, input_shift, output_shift, input_alphabet, output_alphabet diff --git a/docs/shifts/textile.rst b/docs/shifts/textile.rst index 67e6285..a197383 100644 --- a/docs/shifts/textile.rst +++ b/docs/shifts/textile.rst @@ -17,9 +17,9 @@ shift. In [1]: from sofic import TextileSystem - In [2]: from sofic.examples.processes import SlidingNOR + In [2]: from sofic.examples.processes import sliding_nor - In [3]: textile = TextileSystem.from_transducer(SlidingNOR()) + In [3]: textile = TextileSystem.from_transducer(sliding_nor()) @doctest In [4]: textile.induced_code().memory @@ -29,4 +29,4 @@ API === .. autoclass:: TextileSystem - :members: from_transducer, to_transducer, to_sofic_relation, input_shift, output_shift, induced_code + :members: from_transducer, to_transducer, to_product_alphabet_shift, input_shift, output_shift, induced_code diff --git a/docs/shifts/topological_anatomy.rst b/docs/shifts/topological_anatomy.rst index 4d6359e..c309518 100644 --- a/docs/shifts/topological_anatomy.rst +++ b/docs/shifts/topological_anatomy.rst @@ -67,8 +67,8 @@ entropy across both parts, while the sofic even shift is purely bound Out[10]: 0.5527864045001022 The parts add up to :math:`h_\mathrm{top}`, which equals -:meth:`~sofic.shifts.sofic.SoficShift.topological_entropy` divided by -:math:`\ln 2` on a right-resolving presentation. +:meth:`~sofic.shifts.sofic.SoficShift.topological_entropy` (in bits) on a +right-resolving presentation. API === diff --git a/docs/shifts/topological_markov_chain.rst b/docs/shifts/topological_markov_chain.rst index bba680d..e50ffe3 100644 --- a/docs/shifts/topological_markov_chain.rst +++ b/docs/shifts/topological_markov_chain.rst @@ -15,7 +15,7 @@ adjacency matrix. Its Parry measure is the maximum-entropy stochastic generator @doctest float In [2]: tmc.topological_entropy() - Out[2]: 0.48121182505960347 + Out[2]: 0.6942419136306174 API === diff --git a/docs/shifts/wheeler.rst b/docs/shifts/wheeler.rst index 37f3905..0110c37 100644 --- a/docs/shifts/wheeler.rst +++ b/docs/shifts/wheeler.rst @@ -43,10 +43,9 @@ excluded outright, and :func:`wheeler_cover` raises The cover is a :class:`~sofic.shifts.covers.WheelerCover`, a :class:`~sofic.shifts.sofic.SoficShift` subclass that sits beside the Fischer -and Krieger covers. It does not fill the Krieger stubs in -:mod:`sofic.shifts.cover_construction`: a Krieger cover's states are *all* -sets of pasts closed under the follower relation, whereas a Wheeler cover -carries only those that happen to be recency intervals. +and Krieger covers. It is not a Krieger cover: a Krieger cover's states are +*all* follower sets of left-infinite pasts, whereas a Wheeler cover carries +only those that happen to be recency intervals. Once a shift has a Wheeler cover, :func:`wheeler_index_of_shift` gives ``O(|w| log |A|)`` factor-language membership in place of scanning diff --git a/sofic/__init__.py b/sofic/__init__.py index 9d50338..2443353 100644 --- a/sofic/__init__.py +++ b/sofic/__init__.py @@ -37,17 +37,17 @@ WheelerIndex, WheelerOrder, automaton_to_regex, - cartesian_product_gg, - cartesian_product_tt, colex_width, complete, - compose_tg, - compose_tt, + compose_transducer_generator, + compose_transducers, determinize, equivalent, + generator_product, is_wheeler, minimize, transduce_generator, + transducer_product, trim, wheeler_order, ) @@ -86,17 +86,17 @@ wyner_generative_model, ) from sofic.operations import reverse -from sofic.serialization import from_yaml, model_from_yaml, model_to_yaml, read_yaml +from sofic.serialization import model_from_yaml, model_to_yaml, read_yaml from sofic.shifts import ( LeftFischerCover, LeftKriegerCover, MarkovDyckShift, + ProductAlphabetShift, RightFischerCover, RightKriegerCover, ShiftOfFiniteType, SlidingBlockCode, SoficDyckShift, - SoficRelation, SoficShift, SymbolicModel, TextileSystem, @@ -153,7 +153,7 @@ "ShiftOfFiniteType", "SlidingBlockCode", "SoficDyckShift", - "SoficRelation", + "ProductAlphabetShift", "SoficShift", "StateIndex", "StateMachine", @@ -174,17 +174,16 @@ "WheelerOrder", "WynerGenerativeModel", "automaton_to_regex", - "cartesian_product_gg", - "cartesian_product_tt", + "generator_product", + "transducer_product", "colex_width", "complete", - "compose_tg", - "compose_tt", + "compose_transducer_generator", + "compose_transducers", "determinize", "equivalent", "is_wheeler", "minimize", - "from_yaml", "functional_generative_model", "gacs_korner_generative_model", "is_lumpable", diff --git a/sofic/automata/__init__.py b/sofic/automata/__init__.py index 720bf4e..1b7ba55 100644 --- a/sofic/automata/__init__.py +++ b/sofic/automata/__init__.py @@ -1,25 +1,6 @@ """Finite automata and transducers.""" # ``atomaton`` is an intentional pun on atomic automaton. -from sofic.automata.active import ( - EquivalenceOracle, - ExhaustiveEquivalenceOracle, - FunctionMealyOracle, - FunctionMembershipOracle, - LanguageMembershipOracle, - MealyEquivalenceOracle, - MealyExhaustiveEquivalenceOracle, - MealyMembershipOracle, - MembershipOracle, - RandomWalkEquivalenceOracle, - TransducerOutputOracle, - learn_dfa_from_language, - learn_dfa_lstar, - learn_dfa_ttt, - learn_mealy_from_transducer, - learn_mealy_lstar, -) -from sofic.automata.alergia import learn_pfa_alergia from sofic.automata.algorithms import ( MinimizationAlgorithm, complete, @@ -28,30 +9,29 @@ minimize, trim, ) -from sofic.automata.atomaton import Atomaton, AtomicAutomaton, MaximizedPrimeAtomaton from sofic.automata.base import LabeledAutomaton from sofic.automata.buchi import BuchiAutomaton +from sofic.automata.canonical.atomaton import Atomaton, AtomicAutomaton, MaximizedPrimeAtomaton +from sofic.automata.canonical.rfsa import CanonicalRFSA, ResidualFiniteStateAutomaton from sofic.automata.dfa import DFA -from sofic.automata.dfasat import learn_dfa_sat -from sofic.automata.edsm import learn_dfa_edsm -from sofic.automata.icdfa import ( +from sofic.automata.enumeration.icdfa import ( ICDFAString, - count_flag_sequences, count_icdfa, count_icdfa_empty, dfa_to_icdfa_string, first_icdfa_empty_string, - flags_from_string, + icdfa_count_flag_sequences, + icdfa_flags_from_string, + icdfa_next_flags, + icdfa_string_from_flags, icdfa_string_to_dfa, iter_icdfa, iter_icdfa_empty_strings, last_icdfa_empty_string, - next_flags, next_icdfa_empty_string, - string_from_flags, validate_icdfa_empty_string, ) -from sofic.automata.idfa import ( +from sofic.automata.enumeration.idfa import ( MISSING_TRANSITION, count_accessible_idfa, first_idfa_string, @@ -62,20 +42,42 @@ validate_idfa_string, ) from sofic.automata.languages import AutomatonLanguage, RegularLanguage -from sofic.automata.nfa import NFA -from sofic.automata.nwa import NestedWord, NestedWordAutomaton -from sofic.automata.observation import ObservationTable -from sofic.automata.papni import ( +from sofic.automata.learning.active import ( + AutomatonEquivalenceOracle, + EquivalenceOracle, + ExhaustiveEquivalenceOracle, + FunctionMealyOracle, + FunctionMembershipOracle, + LanguageMembershipOracle, + MealyEquivalenceOracle, + MealyExhaustiveEquivalenceOracle, + MealyMembershipOracle, + MembershipOracle, + RandomWalkEquivalenceOracle, + TransducerOutputOracle, + learn_dfa_from_language, + learn_dfa_lstar, + learn_dfa_ttt, + learn_mealy_from_transducer, + learn_mealy_lstar, +) +from sofic.automata.learning.alergia import learn_pfa_alergia +from sofic.automata.learning.dfasat import learn_dfa_sat +from sofic.automata.learning.edsm import learn_dfa_edsm +from sofic.automata.learning.nlstar import learn_prime_atomaton_nlstar, learn_rfsa_from_language, learn_rfsa_nlstar +from sofic.automata.learning.observation import ObservationTable +from sofic.automata.learning.papni import ( DyckAlphabet, + encode_dyck_samples, + encode_dyck_word, is_well_matched, learn_sofic_dyck_shift_papni, - papni_encode, - papni_encode_samples, sofic_dyck_shift_from_papni_dfa, ) +from sofic.automata.learning.rpni import learn_dfa_rpni +from sofic.automata.nfa import NFA +from sofic.automata.nwa import NestedWord, NestedWordAutomaton from sofic.automata.regex import automaton_to_regex -from sofic.automata.rfsa import CanonicalRFSA, ResidualFiniteStateAutomaton -from sofic.automata.rpni import learn_dfa_rpni from sofic.automata.subsequential import ( SubsequentialTransducer, WeightedFiniteStateTransducer, @@ -83,48 +85,42 @@ from sofic.automata.transducer_operations import ( ERROR_STATE, ERROR_SYMBOL, - cartesian_product_gg, - cartesian_product_tt, - compose_tg, - compose_tt, + compose_transducer_generator, + compose_transducers, + generator_product, transduce_generator, + transducer_product, ) from sofic.automata.transducers import MealyMachine, MooreMachine, Transducer from sofic.automata.unifilar import UnifilarAutomaton from sofic.automata.vpa import ( - CallDrivenAutomaton, CanonicalVisiblyPushdownAutomaton, - CompositeVisiblyPushdownAutomaton, DeterministicVisiblyPushdownAutomaton, + ModularVisiblyPushdownAutomaton, MultipleEntryVisiblyPushdownAutomaton, SingleEntryVisiblyPushdownAutomaton, VisiblyPushdownAutomaton, - complement_vpa, - concat_vpa, - difference_vpa, - intersection_vpa, - kleene_star_vpa, - union_vpa, ) from sofic.automata.wheeler import ( WheelerError, WheelerOrder, colex_width, + determinize_wheeler, is_input_consistent, is_wheeler, maximum_colex_relation, - minimum_wdfa, + minimize_wheeler, wheeler_canonical_form, wheeler_isomorphic, wheeler_order, wheeler_state_index, - wnfa_to_wdfa, ) from sofic.automata.wheeler_index import WheelerIndex, wheeler_index __all__ = [ "Atomaton", "AtomicAutomaton", + "AutomatonEquivalenceOracle", "AutomatonLanguage", "BuchiAutomaton", "EquivalenceOracle", @@ -138,10 +134,9 @@ "MembershipOracle", "RandomWalkEquivalenceOracle", "TransducerOutputOracle", - "CallDrivenAutomaton", + "ModularVisiblyPushdownAutomaton", "CanonicalVisiblyPushdownAutomaton", "CanonicalRFSA", - "CompositeVisiblyPushdownAutomaton", "DFA", "DeterministicVisiblyPushdownAutomaton", "DyckAlphabet", @@ -171,26 +166,22 @@ "VisiblyPushdownAutomaton", "automaton_to_regex", "colex_width", - "cartesian_product_gg", - "cartesian_product_tt", - "complement_vpa", - "compose_tg", - "compose_tt", - "concat_vpa", + "generator_product", + "transducer_product", + "compose_transducer_generator", + "compose_transducers", "complete", "count_accessible_idfa", - "count_flag_sequences", + "icdfa_count_flag_sequences", "count_icdfa", "count_icdfa_empty", "determinize", - "difference_vpa", "dfa_to_icdfa_string", "equivalent", "first_icdfa_empty_string", "first_idfa_string", - "flags_from_string", + "icdfa_flags_from_string", "icdfa_string_to_dfa", - "intersection_vpa", "is_input_consistent", "is_well_matched", "is_wheeler", @@ -199,7 +190,6 @@ "iter_idfa_strings", "last_icdfa_empty_string", "MISSING_TRANSITION", - "kleene_star_vpa", "learn_dfa_edsm", "learn_dfa_from_language", "learn_dfa_lstar", @@ -209,22 +199,24 @@ "learn_mealy_from_transducer", "learn_mealy_lstar", "learn_pfa_alergia", + "learn_prime_atomaton_nlstar", + "learn_rfsa_from_language", + "learn_rfsa_nlstar", "learn_sofic_dyck_shift_papni", "maximum_colex_relation", "minimize", - "minimum_wdfa", - "next_flags", + "minimize_wheeler", + "icdfa_next_flags", "next_icdfa_empty_string", - "papni_encode", - "papni_encode_samples", + "encode_dyck_word", + "encode_dyck_samples", "rank_idfa_string", "reroot_idfa_string", "sofic_dyck_shift_from_papni_dfa", - "string_from_flags", + "icdfa_string_from_flags", "trim", "transduce_generator", "unrank_idfa_string", - "union_vpa", "validate_idfa_string", "validate_icdfa_empty_string", "wheeler_canonical_form", @@ -232,5 +224,5 @@ "wheeler_isomorphic", "wheeler_order", "wheeler_state_index", - "wnfa_to_wdfa", + "determinize_wheeler", ] diff --git a/sofic/automata/_words.py b/sofic/automata/_words.py new file mode 100644 index 0000000..1f631a9 --- /dev/null +++ b/sofic/automata/_words.py @@ -0,0 +1,23 @@ +"""Shared word enumeration for automata modules.""" + +from __future__ import annotations + +from collections.abc import Iterable, Iterator +from typing import Any + +Word = tuple[Any, ...] + + +def _words_up_to(max_length: int, alphabet: Iterable[Any]) -> Iterator[Word]: + """Yield every word of length at most ``max_length`` in breadth-first order.""" + symbols = tuple(alphabet) + frontier: list[Word] = [()] + yield () + for _ in range(max_length): + nxt: list[Word] = [] + for word in frontier: + for symbol in symbols: + extended = (*word, symbol) + yield extended + nxt.append(extended) + frontier = nxt diff --git a/sofic/automata/algorithms.py b/sofic/automata/algorithms.py index bb58d58..b7b41a6 100644 --- a/sofic/automata/algorithms.py +++ b/sofic/automata/algorithms.py @@ -244,9 +244,15 @@ def minimize_hopcroft(dfa: DFA, *, alphabet: frozenset[Any] | None = None) -> DF def equivalent( aut1: LabeledAutomaton, aut2: LabeledAutomaton, - alphabet: frozenset[Any], + alphabet: frozenset[Any] | None = None, ) -> bool: - """Return whether two automata recognize the same language over ``alphabet``.""" + """Return whether two automata recognize the same language. + + The comparison runs over ``alphabet`` together with every symbol either + automaton declares or uses, so a too-small ``alphabet`` cannot hide a + difference on the omitted symbols. + """ + alphabet = frozenset(alphabet or ()) | _transition_alphabet(aut1) | _transition_alphabet(aut2) d1 = minimize(_to_nfa(aut1), alphabet=alphabet, algorithm="hopcroft") d2 = minimize(_to_nfa(aut2), alphabet=alphabet, algorithm="hopcroft") return _isomorphic_minimal_dfa(d1, d2, alphabet) @@ -281,6 +287,15 @@ def _effective_alphabet(aut: LabeledAutomaton) -> frozenset[Any]: return frozenset(symbols) +def _transition_alphabet(aut: LabeledAutomaton) -> frozenset[Any]: + symbols = {symbol for symbol in aut.input_alphabet if symbol is not EPSILON} + for transition in aut.transitions(): + symbol = transition.data.get(ATTR_SYMBOL) + if symbol is not None and symbol is not EPSILON: + symbols.add(symbol) + return frozenset(symbols) + + def _forward_reachable(aut: LabeledAutomaton) -> set[Hashable]: if not aut.initial_states: return set() diff --git a/sofic/automata/base.py b/sofic/automata/base.py index f107546..6c3a226 100644 --- a/sofic/automata/base.py +++ b/sofic/automata/base.py @@ -47,7 +47,7 @@ def recognizes(self, word: Sequence[Any]) -> bool: def words_of_length(self, length: int) -> Iterator[tuple[Any, ...]]: """Yield accepted words of exactly ``length`` symbols.""" - from sofic.automata.enumeration import words_of_length + from sofic.automata.enumeration.words import words_of_length yield from words_of_length(self, length) @@ -57,7 +57,7 @@ def iter_language(self, max_length: int | None = None) -> Iterator[tuple[Any, .. If ``max_length`` is omitted, the iterator is unbounded and may not terminate for finite languages after yielding their last word. """ - from sofic.automata.enumeration import iter_language + from sofic.automata.enumeration.words import iter_language yield from iter_language(self, max_length=max_length) @@ -79,10 +79,6 @@ def intersection(self, other: LabeledAutomaton) -> DFA: return intersection_dfa(self, other) - def intersect(self, other: LabeledAutomaton) -> DFA: - """Alias for :meth:`intersection`.""" - return self.intersection(other) - def complement(self, alphabet: frozenset[Any] | None = None) -> DFA: """Return a complete DFA recognizing the complement over ``alphabet``.""" from sofic.automata.languages.automaton_ops import complement_dfa @@ -101,20 +97,12 @@ def concat(self, other: LabeledAutomaton) -> NFA: return concat_nfa(self, other) - def concatenate(self, other: LabeledAutomaton) -> NFA: - """Alias for :meth:`concat`.""" - return self.concat(other) - def kleene_star(self) -> NFA: """Return an NFA recognizing the Kleene star of this language.""" from sofic.automata.languages.automaton_ops import kleene_star_nfa return kleene_star_nfa(self) - def star(self) -> NFA: - """Alias for :meth:`kleene_star`.""" - return self.kleene_star() - def is_deterministic(self) -> bool: """Return whether this automaton is DFA-deterministic.""" from sofic.properties import is_deterministic_automaton @@ -140,14 +128,7 @@ def epsilon_closure(self, states: set[Hashable]) -> set[Hashable]: return closure def _run_nfa(self, word: Sequence[Any], start: set[Hashable] | None = None) -> set[Hashable]: - seed = set(self.initial_states) if start is None else set(start) - current = self.epsilon_closure(seed) - for symbol in word: - next_states: set[Hashable] = set() - for state in current: - next_states.update(self.delta(state, symbol)) - current = self.epsilon_closure(next_states) - return current + return run_nfa(self, word, start=start) def reverse(self) -> NFA: """Return an NFA recognizing the reversed language.""" @@ -159,3 +140,22 @@ def reverse(self) -> NFA: accepting_states=frozenset(self.epsilon_closure(set(self.initial_states))), graph=self.graph.reverse(), ) + + +def run_nfa( + aut: LabeledAutomaton, + word: Sequence[Any], + start: set[Hashable] | frozenset[Hashable] | None = None, +) -> set[Hashable]: + """Return the epsilon-closed state set reached by reading ``word``. + + Simulation starts from ``start`` (default: ``aut.initial_states``). + """ + seed = set(aut.initial_states) if start is None else set(start) + current = aut.epsilon_closure(seed) + for symbol in word: + next_states: set[Hashable] = set() + for state in current: + next_states.update(aut.delta(state, symbol)) + current = aut.epsilon_closure(next_states) + return current diff --git a/sofic/automata/buchi_simulation.py b/sofic/automata/buchi_simulation.py index 26da207..f6f9973 100644 --- a/sofic/automata/buchi_simulation.py +++ b/sofic/automata/buchi_simulation.py @@ -9,7 +9,12 @@ def accepts_lasso_buchi(ba: BuchiAutomaton, prefix: Sequence[Any], loop: Sequence[Any]) -> bool: - """Accept if repeating ``loop`` after ``prefix`` visits an accepting state infinitely often.""" + """Accept if repeating ``loop`` after ``prefix`` visits an accepting state infinitely often. + + ``loop`` must be non-empty: ``prefix loop^omega`` is an infinite word only then. + """ + if not loop: + raise ValueError("an ultimately periodic omega-word needs a non-empty loop") post = ba._run_nfa(prefix) if not post: return False diff --git a/sofic/automata/canonical/__init__.py b/sofic/automata/canonical/__init__.py new file mode 100644 index 0000000..7b3bb5a --- /dev/null +++ b/sofic/automata/canonical/__init__.py @@ -0,0 +1,41 @@ +"""Canonical residual automata, atomata, and their duality.""" + +from sofic.automata.canonical.atomaton import ( + Atomaton, + AtomicAutomaton, + MaximizedPrimeAtomaton, + atomic_states, + is_atomic, +) +from sofic.automata.canonical.dual import dual_atomaton_from_rfsa, dual_rfsa_from_atomaton +from sofic.automata.canonical.residual import ( + ResidualTable, + atomaton_from_language, + canonical_rfsa_from_language, + maximized_prime_atomaton_from_language, + observation_to_atomaton, + observation_to_canonical_rfsa, + observation_to_maximized_prime_atomaton, + observation_to_minimal_dfa, +) +from sofic.automata.canonical.rfsa import CanonicalRFSA, ResidualFiniteStateAutomaton + +__all__ = [ + "Atomaton", + "AtomicAutomaton", + "CanonicalRFSA", + "MaximizedPrimeAtomaton", + "ResidualFiniteStateAutomaton", + "ResidualTable", + "atomaton_from_language", + "atomic_states", + "canonical_rfsa_from_language", + "dual_atomaton_from_rfsa", + "dual_rfsa_from_atomaton", + "is_atomic", + "maximized_prime_atomaton_from_language", + "observation_to_atomaton", + "observation_to_canonical_rfsa", + "observation_to_maximized_prime_atomaton", + "observation_to_minimal_dfa", +] diff --git a/sofic/automata/atomaton.py b/sofic/automata/canonical/atomaton.py similarity index 72% rename from sofic/automata/atomaton.py rename to sofic/automata/canonical/atomaton.py index 51f9862..7e22abe 100644 --- a/sofic/automata/atomaton.py +++ b/sofic/automata/canonical/atomaton.py @@ -11,8 +11,8 @@ from sofic.exceptions import SoficValidationError if TYPE_CHECKING: - from sofic.automata.observation import ObservationTable - from sofic.automata.rfsa import CanonicalRFSA + from sofic.automata.canonical.rfsa import CanonicalRFSA + from sofic.automata.learning.observation import ObservationTable def atomic_states(nfa: NFA, *, alphabet: frozenset[Any] | None = None) -> frozenset[Hashable]: @@ -70,7 +70,7 @@ class Atomaton(AtomicAutomaton): @classmethod def from_language(cls, language: RegularLanguage | NFA, **kwargs: Any) -> Atomaton: - from sofic.automata.canonical_extraction import atomaton_from_language + from sofic.automata.canonical.residual import atomaton_from_language return atomaton_from_language(language) @@ -80,23 +80,37 @@ def to_minimal_dfa_via_double_reversal(self) -> DFA: return minimize(self, algorithm="brzozowski") -class MaximizedPrimeAtomaton(AtomicAutomaton): - """Maximized prime átomaton — dual of the canonical RFSA.""" +class MaximizedPrimeAtomaton(NFA): + """Maximized prime átomaton: the dual of the canonical RFSA :cite:`MaarandTamm2022`. + + It is the reverse of the canonical RFSA of the reversed language, just as the + átomaton is the reverse of the minimal DFA of the reversed language. Its + states are the maximized prime atoms; the right language of each lies between + its atom and its maximized atom :cite:`Tamm2015`, so -- unlike the átomaton -- + it need not be atomic. + """ @classmethod def from_language(cls, language: RegularLanguage | NFA, **kwargs: Any) -> MaximizedPrimeAtomaton: - from sofic.automata.canonical_extraction import maximized_prime_atomaton_from_language + from sofic.automata.canonical.residual import maximized_prime_atomaton_from_language return maximized_prime_atomaton_from_language(language) @classmethod def from_observation_table(cls, table: ObservationTable, **kwargs: Any) -> MaximizedPrimeAtomaton: - from sofic.automata.canonical_extraction import observation_to_maximized_prime_atomaton + from sofic.automata.canonical.residual import observation_to_maximized_prime_atomaton return observation_to_maximized_prime_atomaton(table) @classmethod def from_canonical_rfsa(cls, rfsa: CanonicalRFSA, **kwargs: Any) -> MaximizedPrimeAtomaton: - from sofic.automata.canonical_dual import dual_atomaton_from_rfsa + """Return the maximized prime átomaton of the language ``rfsa`` recognizes.""" + from sofic.automata.canonical.residual import maximized_prime_atomaton_from_language + + return maximized_prime_atomaton_from_language(rfsa) + + def dual(self) -> CanonicalRFSA: + """Return the reverse automaton: the canonical RFSA of the reversed language.""" + from sofic.automata.canonical.dual import dual_rfsa_from_atomaton - return dual_atomaton_from_rfsa(rfsa) + return dual_rfsa_from_atomaton(self) diff --git a/sofic/automata/canonical/dual.py b/sofic/automata/canonical/dual.py new file mode 100644 index 0000000..4a4aaf4 --- /dev/null +++ b/sofic/automata/canonical/dual.py @@ -0,0 +1,25 @@ +"""Duality between canonical RFSAs and maximized prime átomata. + +The maximized prime átomaton of ``L`` is the reverse of the canonical RFSA of +the reversed language :cite:`MaarandTamm2022`. Reversing either object therefore gives +the other one for the reversed language, and reversing twice is the identity. +""" + +from __future__ import annotations + +from sofic.automata.canonical.atomaton import MaximizedPrimeAtomaton +from sofic.automata.canonical.rfsa import CanonicalRFSA + + +def dual_atomaton_from_rfsa(rfsa: CanonicalRFSA) -> MaximizedPrimeAtomaton: + """Reverse the canonical RFSA of ``L`` into the maximized prime átomaton of ``L^R``.""" + from sofic.automata.canonical.residual import _reverse_into + + return _reverse_into(MaximizedPrimeAtomaton, rfsa) + + +def dual_rfsa_from_atomaton(atomaton: MaximizedPrimeAtomaton) -> CanonicalRFSA: + """Reverse the maximized prime átomaton of ``L`` into the canonical RFSA of ``L^R``.""" + from sofic.automata.canonical.residual import _reverse_into + + return _reverse_into(CanonicalRFSA, atomaton) diff --git a/sofic/automata/canonical/residual.py b/sofic/automata/canonical/residual.py new file mode 100644 index 0000000..b463e38 --- /dev/null +++ b/sofic/automata/canonical/residual.py @@ -0,0 +1,318 @@ +"""Canonical automaton extraction from languages and observation tables. + +Every construction works on the minimal complete DFA ``D`` of the language +``L``. Its states are the residuals (left quotients) ``u^{-1} L``, and inclusion +and union questions between residuals are decided exactly on ``D``: + +* ``L_p <= L_q`` holds iff no pair reachable from ``(p, q)`` in ``D x D`` is + accepting in the first component only; +* ``L_q <= union(L_p for p in S)`` holds iff no pair reachable from ``(q, S)`` in + ``D x subsets(D)`` accepts on the left with no accepting state on the right. + +A residual is *prime* when it is non-empty and not the union of the residuals +strictly contained in it :cite:`Denis2002`. +""" + +from __future__ import annotations + +from collections import deque +from collections.abc import Hashable, Iterable +from dataclasses import dataclass +from typing import Any + +from sofic.automata.base import LabeledAutomaton +from sofic.automata.canonical.atomaton import Atomaton, MaximizedPrimeAtomaton +from sofic.automata.canonical.rfsa import CanonicalRFSA +from sofic.automata.dfa import DFA +from sofic.automata.languages.base import AutomatonLanguage, RegularLanguage, as_language +from sofic.automata.learning.observation import ObservationTable +from sofic.automata.nfa import NFA +from sofic.graph import ATTR_SYMBOL +from sofic.states import sequential_labels + + +def _language_automaton(language: RegularLanguage | LabeledAutomaton) -> LabeledAutomaton: + if isinstance(language, LabeledAutomaton): + return language + lang = as_language(language) # type: ignore[arg-type] + if isinstance(lang, AutomatonLanguage): + return lang.automaton + raise TypeError(f"cannot extract automaton from {type(language)!r}") + + +@dataclass(frozen=True) +class ResidualTable: + """Minimal complete DFA of a language, viewed as its table of residuals.""" + + dfa: DFA + alphabet: tuple[Any, ...] + start: Hashable + accepting: frozenset[Hashable] + delta: dict[tuple[Hashable, Any], Hashable] + + @classmethod + def from_automaton(cls, aut: LabeledAutomaton) -> ResidualTable: + from sofic.automata.algorithms import _transition_alphabet, complete, minimize + + symbols = _transition_alphabet(aut) + source = aut if isinstance(aut, (NFA, DFA)) else NFA(**_automaton_kwargs(aut)) + dfa = complete(minimize(source, alphabet=symbols), symbols) + if not dfa.initial_states: + empty = DFA(input_alphabet=frozenset(symbols), initial_states=frozenset({"empty"})) + empty.graph.add_state("empty") + dfa = complete(empty, symbols) + delta = { + (transition.source, transition.data[ATTR_SYMBOL]): transition.target for transition in dfa.transitions() + } + return cls( + dfa=dfa, + alphabet=tuple(sorted(symbols, key=repr)), + start=next(iter(dfa.initial_states)), + accepting=frozenset(dfa.accepting_states), + delta=delta, + ) + + @property + def states(self) -> tuple[Hashable, ...]: + return tuple(self.dfa.states()) + + def is_empty(self, state: Hashable) -> bool: + """Whether the residual ``L_state`` is empty.""" + seen = {state} + queue = deque([state]) + while queue: + current = queue.popleft() + if current in self.accepting: + return False + for symbol in self.alphabet: + target = self.delta[(current, symbol)] + if target not in seen: + seen.add(target) + queue.append(target) + return True + + def includes(self, smaller: Hashable, larger: Hashable) -> bool: + """Whether ``L_smaller <= L_larger``.""" + seen = {(smaller, larger)} + queue = deque(seen) + while queue: + left, right = queue.popleft() + if left in self.accepting and right not in self.accepting: + return False + for symbol in self.alphabet: + pair = (self.delta[(left, symbol)], self.delta[(right, symbol)]) + if pair not in seen: + seen.add(pair) + queue.append(pair) + return True + + def is_covered(self, state: Hashable, cover: Iterable[Hashable]) -> bool: + """Whether ``L_state`` is contained in the union of ``L_p`` for ``p`` in ``cover``.""" + start = (state, frozenset(cover)) + seen = {start} + queue = deque([start]) + while queue: + left, right = queue.popleft() + if left in self.accepting and not (right & self.accepting): + return False + for symbol in self.alphabet: + pair = (self.delta[(left, symbol)], frozenset(self.delta[(q, symbol)] for q in right)) + if pair not in seen: + seen.add(pair) + queue.append(pair) + return True + + def prime_states(self) -> tuple[Hashable, ...]: + """States whose residual is prime, in breadth-first order from the start.""" + nonempty = [q for q in self._bfs_order() if not self.is_empty(q)] + primes = [] + for q in nonempty: + strictly_smaller = [p for p in nonempty if p != q and self.includes(p, q)] + if not self.is_covered(q, strictly_smaller): + primes.append(q) + return tuple(primes) + + def _bfs_order(self) -> list[Hashable]: + order = [self.start] + seen = {self.start} + for state in order: + for symbol in self.alphabet: + target = self.delta[(state, symbol)] + if target not in seen: + seen.add(target) + order.append(target) + return order + + def residual_automaton(self, state: Hashable) -> DFA: + """DFA recognizing the residual ``L_state``.""" + result = self.dfa.copy() + result.initial_states = frozenset({state}) + return result + + def left_language_automaton(self, state: Hashable) -> DFA: + """DFA recognizing the words that lead from the start to ``state``.""" + result = self.dfa.copy() + result.accepting_states = frozenset({state}) + return result + + +def _automaton_kwargs(aut: LabeledAutomaton) -> dict[str, Any]: + return { + "input_alphabet": aut.input_alphabet, + "initial_states": aut.initial_states, + "accepting_states": aut.accepting_states, + "graph": aut.graph.copy(), + } + + +def _labels(count: int) -> tuple[Hashable, ...]: + return sequential_labels(count) if count <= 26 else tuple(range(count)) + + +def canonical_rfsa_from_language(language: RegularLanguage | LabeledAutomaton) -> CanonicalRFSA: + r"""Build the canonical residual finite-state automaton of a regular language. + + Following :cite:`Denis2002`, the states are the prime residuals of ``L``; the + initial states are the primes contained in ``L``; the accepting states are + the primes containing the empty word; and there is a transition + :math:`p \xrightarrow{a} p'` exactly when :math:`L_{p'} \subseteq a^{-1} L_p`. + The canonical RFSA is saturated (it has every such transition) and is never + larger than the minimal DFA, often exponentially smaller. + """ + table = ResidualTable.from_automaton(_language_automaton(language)) + primes = table.prime_states() + name = dict(zip(primes, _labels(len(primes)), strict=True)) + rfsa = CanonicalRFSA( + input_alphabet=frozenset(table.alphabet), + initial_states=frozenset(name[p] for p in primes if table.includes(p, table.start)), + accepting_states=frozenset(name[p] for p in primes if p in table.accepting), + ) + for p in primes: + rfsa.graph.add_state(name[p]) + for p in primes: + for symbol in table.alphabet: + successor = table.delta[(p, symbol)] + for target in primes: + if table.includes(target, successor): + rfsa.add_transition(name[p], name[target], symbol) + return rfsa + + +def atomaton_from_language(language: RegularLanguage | LabeledAutomaton) -> Atomaton: + """Build átomaton via double-reversal pipeline. + + The átomaton is the *reverse of the minimal DFA of the reverse language* + (:cite:`BrzozowskiTamm2014`, Theorem 2), so the pipeline must stop at the + reversal: determinizing once more would collapse it back to the minimal DFA + of ``language``, which is Brzozowski's minimization rather than the + átomaton. + """ + from sofic.automata.languages.automaton_ops import minimal_dfa_from_language + + aut = _language_automaton(language) + dfa = minimal_dfa_from_language(aut) + rev = dfa.reverse().determinize().minimize() + atom = rev.reverse() + return Atomaton( + input_alphabet=atom.input_alphabet, + initial_states=atom.initial_states, + accepting_states=atom.accepting_states, + graph=atom.graph.copy(), + ) + + +def maximized_prime_atomaton_from_language( + language: RegularLanguage | LabeledAutomaton, +) -> MaximizedPrimeAtomaton: + r"""Build the maximized prime átomaton of a regular language :cite:`MaarandTamm2022`. + + It is the reverse of the canonical RFSA of :math:`L^R`. Equivalently, it is + the subautomaton of the maximized átomaton of ``L`` -- the reverse of the + saturated minimal DFA of :math:`L^R` -- on the maximized atoms whose + quotients of :math:`L^R` are prime :cite:`Tamm2015`. + """ + reversed_rfsa = canonical_rfsa_from_language(_language_automaton(language).reverse()) + return _reverse_into(MaximizedPrimeAtomaton, reversed_rfsa) + + +def _reverse_into(cls: type[NFA], aut: NFA) -> Any: + reversed_aut = aut.reverse() + return cls( + input_alphabet=reversed_aut.input_alphabet, + initial_states=reversed_aut.initial_states, + accepting_states=reversed_aut.accepting_states, + graph=reversed_aut.graph.copy(), + ) + + +def observation_to_minimal_dfa(table: ObservationTable) -> DFA: + """Angluin-style DFA extraction from a closed, consistent table.""" + access = sorted(table.access_words, key=lambda w: (len(w), w)) + experiments = sorted(table.experiments, key=lambda w: (len(w), w)) + + def signature(word: tuple[Any, ...]) -> tuple[bool, ...]: + return tuple(table.membership.get(word + exp, False) for exp in experiments) + + classes: dict[tuple[bool, ...], tuple[Any, ...]] = {} + for word in access: + sig = signature(word) + classes.setdefault(sig, word) + + state_for_word = {word: classes[signature(word)] for word in access} + alphabet = _infer_alphabet(table) + dfa = DFA( + input_alphabet=alphabet, + initial_states=frozenset({()}), + accepting_states=frozenset(), + ) + for word in access: + dfa.graph.add_state(state_for_word[word]) + for word in access: + for symbol in alphabet: + target_word = word + (symbol,) + if target_word in state_for_word: + dfa.add_transition(state_for_word[word], state_for_word[target_word], symbol) + accepting = {state_for_word[word] for word in access if table.membership.get(word, False)} + dfa.accepting_states = frozenset(accepting) + return dfa + + +def observation_to_canonical_rfsa(table: ObservationTable) -> CanonicalRFSA: + r"""NL\*-style RFSA extraction from an RFSA-closed, RFSA-consistent table. + + States are the rows of the access words that are prime among all rows of + the table, exactly as in the NL\* hypothesis :cite:`Bollig2009`. + """ + from sofic.automata.learning.nlstar import _hypothesis, _primes + + experiments = sorted(table.experiments, key=lambda w: (len(w), w)) + if () in experiments: + experiments.remove(()) + experiments.insert(0, ()) + symbols = tuple(sorted(_infer_alphabet(table), key=repr)) + access = sorted(table.access_words, key=lambda w: (len(w), w)) + + def row(word: tuple[Any, ...]) -> tuple[bool, ...]: + return tuple(table.membership.get(word + exp, False) for exp in experiments) + + upper = {u: row(u) for u in access} + lower = {u + (a,): row(u + (a,)) for u in access for a in symbols} + primes_upper = _primes(set(upper.values()) | set(lower.values()), len(experiments)) & set(upper.values()) + return _hypothesis(access, symbols, upper, primes_upper, row) + + +def observation_to_atomaton(table: ObservationTable) -> Atomaton: + dfa = observation_to_minimal_dfa(table) + return atomaton_from_language(dfa) + + +def observation_to_maximized_prime_atomaton(table: ObservationTable) -> MaximizedPrimeAtomaton: + dfa = observation_to_minimal_dfa(table) + return maximized_prime_atomaton_from_language(dfa) + + +def _infer_alphabet(table: ObservationTable) -> frozenset[Any]: + symbols: set[Any] = set() + for word in table.access_words | table.experiments: + symbols.update(word) + return frozenset(symbols) diff --git a/sofic/automata/canonical/rfsa.py b/sofic/automata/canonical/rfsa.py new file mode 100644 index 0000000..1a0bf48 --- /dev/null +++ b/sofic/automata/canonical/rfsa.py @@ -0,0 +1,57 @@ +"""Residual and canonical RFSA automata.""" + +from __future__ import annotations + +from typing import TYPE_CHECKING, Any + +from sofic.automata.languages.base import RegularLanguage +from sofic.automata.nfa import NFA + +if TYPE_CHECKING: + from sofic.automata.canonical.atomaton import MaximizedPrimeAtomaton + from sofic.automata.learning.observation import ObservationTable + + +class ResidualFiniteStateAutomaton(NFA): + """NFA whose states accept residual languages of the recognized language :cite:`Denis2002`.""" + + def validate(self) -> None: + super().validate() + from sofic.automata.algorithms import equivalent + from sofic.automata.canonical.residual import ResidualTable + + if not self.initial_states: + return + table = ResidualTable.from_automaton(self) + residuals = [table.residual_automaton(q) for q in table.states] + for state in self.states(): + right = NFA( + input_alphabet=self.input_alphabet, + initial_states=frozenset({state}), + accepting_states=self.accepting_states, + graph=self.graph, + ) + if not any(equivalent(right, residual, frozenset(table.alphabet)) for residual in residuals): + self._require(False, f"right language of state {state!r} is not a residual of the language") + + +class CanonicalRFSA(ResidualFiniteStateAutomaton): + """Canonical residual finite-state automaton R(L) :cite:`Denis2002`.""" + + @classmethod + def from_language(cls, language: RegularLanguage | NFA, **kwargs: Any) -> CanonicalRFSA: + from sofic.automata.canonical.residual import canonical_rfsa_from_language + + return canonical_rfsa_from_language(language) + + @classmethod + def from_observation_table(cls, table: ObservationTable, **kwargs: Any) -> CanonicalRFSA: + from sofic.automata.canonical.residual import observation_to_canonical_rfsa + + return observation_to_canonical_rfsa(table) + + def dual(self) -> MaximizedPrimeAtomaton: + """Return the reverse automaton: the maximized prime átomaton of the reversed language.""" + from sofic.automata.canonical.dual import dual_atomaton_from_rfsa + + return dual_atomaton_from_rfsa(self) diff --git a/sofic/automata/canonical_dual.py b/sofic/automata/canonical_dual.py deleted file mode 100644 index b8b7d43..0000000 --- a/sofic/automata/canonical_dual.py +++ /dev/null @@ -1,18 +0,0 @@ -"""Dual constructions between canonical RFSA and maximized prime átomaton.""" - -from __future__ import annotations - -from sofic.automata.atomaton import MaximizedPrimeAtomaton -from sofic.automata.rfsa import CanonicalRFSA - - -def dual_atomaton_from_rfsa(rfsa: CanonicalRFSA) -> MaximizedPrimeAtomaton: - from sofic.automata.canonical_extraction import maximized_prime_atomaton_from_language - - return maximized_prime_atomaton_from_language(rfsa) - - -def dual_rfsa_from_atomaton(atomaton: MaximizedPrimeAtomaton) -> CanonicalRFSA: - from sofic.automata.canonical_extraction import canonical_rfsa_from_language - - return canonical_rfsa_from_language(atomaton) diff --git a/sofic/automata/canonical_extraction.py b/sofic/automata/canonical_extraction.py deleted file mode 100644 index 533f913..0000000 --- a/sofic/automata/canonical_extraction.py +++ /dev/null @@ -1,129 +0,0 @@ -"""Canonical automaton extraction from languages and observation tables.""" - -from __future__ import annotations - -from typing import Any - -from sofic.automata.atomaton import Atomaton, MaximizedPrimeAtomaton -from sofic.automata.dfa import DFA -from sofic.automata.languages.automaton_ops import ( - minimal_dfa_from_language, -) -from sofic.automata.languages.base import AutomatonLanguage, RegularLanguage, as_language -from sofic.automata.nfa import NFA -from sofic.automata.observation import ObservationTable -from sofic.automata.rfsa import CanonicalRFSA - - -def _language_automaton(language: RegularLanguage | NFA | DFA) -> NFA | DFA: - if isinstance(language, (NFA, DFA)): - return language - lang = as_language(language) # type: ignore[arg-type] - if isinstance(lang, AutomatonLanguage): - return lang.automaton - raise TypeError(f"cannot extract automaton from {type(language)!r}") - - -def canonical_rfsa_from_language(language: RegularLanguage | NFA | DFA) -> CanonicalRFSA: - """Build canonical RFSA from prime residuals of the language. - - Phase 2 placeholder: currently reuses the minimal DFA structure directly. - """ - aut = _language_automaton(language) - dfa = minimal_dfa_from_language(aut) - return CanonicalRFSA( - input_alphabet=dfa.input_alphabet, - initial_states=dfa.initial_states, - accepting_states=dfa.accepting_states, - graph=dfa.graph.copy(), - ) - - -def atomaton_from_language(language: RegularLanguage | NFA | DFA) -> Atomaton: - """Build átomaton via double-reversal pipeline. - - The átomaton is the *reverse of the minimal DFA of the reverse language* - (:cite:`BrzozowskiTamm2014`, Theorem 2), so the pipeline must stop at the - reversal: determinizing once more would collapse it back to the minimal DFA - of ``language``, which is Brzozowski's minimization rather than the - átomaton. - """ - aut = _language_automaton(language) - dfa = minimal_dfa_from_language(aut) - rev = dfa.reverse().determinize().minimize() - atom = rev.reverse() - return Atomaton( - input_alphabet=atom.input_alphabet, - initial_states=atom.initial_states, - accepting_states=atom.accepting_states, - graph=atom.graph.copy(), - ) - - -def maximized_prime_atomaton_from_language(language: RegularLanguage | NFA | DFA) -> MaximizedPrimeAtomaton: - """Build maximized prime átomaton from a language. - - Phase 2 placeholder: currently reuses the minimal DFA structure directly. - """ - aut = _language_automaton(language) - lang = AutomatonLanguage(minimal_dfa_from_language(aut)) - return MaximizedPrimeAtomaton( - input_alphabet=lang.automaton.input_alphabet, - initial_states=lang.automaton.initial_states, - accepting_states=lang.automaton.accepting_states, - graph=lang.automaton.graph.copy(), - ) - - -def observation_to_minimal_dfa(table: ObservationTable) -> DFA: - """Angluin-style DFA extraction from a closed, consistent table.""" - access = sorted(table.access_words, key=lambda w: (len(w), w)) - experiments = sorted(table.experiments, key=lambda w: (len(w), w)) - - def signature(word: tuple[Any, ...]) -> tuple[bool, ...]: - return tuple(table.membership.get(word + exp, False) for exp in experiments) - - classes: dict[tuple[bool, ...], tuple[Any, ...]] = {} - for word in access: - sig = signature(word) - classes.setdefault(sig, word) - - state_for_word = {word: classes[signature(word)] for word in access} - alphabet = _infer_alphabet(table) - dfa = DFA( - input_alphabet=alphabet, - initial_states=frozenset({()}), - accepting_states=frozenset(), - ) - for word in access: - dfa.graph.add_state(state_for_word[word]) - for word in access: - for symbol in alphabet: - target_word = word + (symbol,) - if target_word in state_for_word: - dfa.add_transition(state_for_word[word], state_for_word[target_word], symbol) - accepting = {state_for_word[word] for word in access if table.membership.get(word, False)} - dfa.accepting_states = frozenset(accepting) - return dfa - - -def observation_to_canonical_rfsa(table: ObservationTable) -> CanonicalRFSA: - dfa = observation_to_minimal_dfa(table) - return canonical_rfsa_from_language(dfa) - - -def observation_to_atomaton(table: ObservationTable) -> Atomaton: - dfa = observation_to_minimal_dfa(table) - return atomaton_from_language(dfa) - - -def observation_to_maximized_prime_atomaton(table: ObservationTable) -> MaximizedPrimeAtomaton: - dfa = observation_to_minimal_dfa(table) - return maximized_prime_atomaton_from_language(dfa) - - -def _infer_alphabet(table: ObservationTable) -> frozenset[Any]: - symbols: set[Any] = set() - for word in table.access_words | table.experiments: - symbols.update(word) - return frozenset(symbols) diff --git a/sofic/automata/enumeration/__init__.py b/sofic/automata/enumeration/__init__.py new file mode 100644 index 0000000..26372de --- /dev/null +++ b/sofic/automata/enumeration/__init__.py @@ -0,0 +1,58 @@ +"""Enumeration of words and of initially connected / accessible DFAs.""" + +from sofic.automata.enumeration.icdfa import ( + ICDFAString, + count_icdfa, + count_icdfa_empty, + dfa_to_icdfa_string, + first_icdfa_empty_string, + icdfa_count_flag_sequences, + icdfa_flags_from_string, + icdfa_next_flags, + icdfa_string_from_flags, + icdfa_string_to_dfa, + iter_icdfa, + iter_icdfa_empty_strings, + last_icdfa_empty_string, + next_icdfa_empty_string, + validate_icdfa_empty_string, +) +from sofic.automata.enumeration.idfa import ( + MISSING_TRANSITION, + count_accessible_idfa, + first_idfa_string, + iter_idfa_strings, + rank_idfa_string, + reroot_idfa_string, + unrank_idfa_string, + validate_idfa_string, +) +from sofic.automata.enumeration.words import iter_language, words_of_length + +__all__ = [ + "ICDFAString", + "MISSING_TRANSITION", + "count_accessible_idfa", + "icdfa_count_flag_sequences", + "count_icdfa", + "count_icdfa_empty", + "dfa_to_icdfa_string", + "first_icdfa_empty_string", + "first_idfa_string", + "icdfa_flags_from_string", + "icdfa_string_to_dfa", + "iter_icdfa", + "iter_icdfa_empty_strings", + "iter_idfa_strings", + "iter_language", + "last_icdfa_empty_string", + "icdfa_next_flags", + "next_icdfa_empty_string", + "rank_idfa_string", + "reroot_idfa_string", + "icdfa_string_from_flags", + "unrank_idfa_string", + "validate_icdfa_empty_string", + "validate_idfa_string", + "words_of_length", +] diff --git a/sofic/automata/icdfa.py b/sofic/automata/enumeration/icdfa.py similarity index 95% rename from sofic/automata/icdfa.py rename to sofic/automata/enumeration/icdfa.py index 8723930..68e525c 100644 --- a/sofic/automata/icdfa.py +++ b/sofic/automata/enumeration/icdfa.py @@ -19,19 +19,19 @@ __all__ = [ "ICDFAString", - "count_flag_sequences", + "icdfa_count_flag_sequences", "count_icdfa", "count_icdfa_empty", "dfa_to_icdfa_string", "first_icdfa_empty_string", - "flags_from_string", + "icdfa_flags_from_string", "icdfa_string_to_dfa", "iter_icdfa", "iter_icdfa_empty_strings", "last_icdfa_empty_string", - "next_flags", + "icdfa_next_flags", "next_icdfa_empty_string", - "string_from_flags", + "icdfa_string_from_flags", "validate_icdfa_empty_string", ] @@ -79,7 +79,7 @@ def validate_icdfa_empty_string( raise ICDFAEnumerationError(f"state {state} does not appear in the first {k * state} symbols") -def flags_from_string(transitions: Sequence[int], *, n: int) -> tuple[int, ...]: +def icdfa_flags_from_string(transitions: Sequence[int], *, n: int) -> tuple[int, ...]: """Return first-occurrence indices ``(f_1, …, f_{n-1})`` for ``transitions``.""" if n <= 1: return () @@ -104,7 +104,7 @@ def _validate_flags(flags: Sequence[int], *, n: int, k: int) -> None: raise ICDFAEnumerationError(f"flag f_{index + 1}={flags[index]} not in ({lower}, {upper}]") -def string_from_flags( +def icdfa_string_from_flags( flags: Sequence[int], *, n: int, @@ -124,7 +124,7 @@ def first_icdfa_empty_string(*, n: int, k: int) -> tuple[int, ...]: if n == 1: return (0,) * k flags = tuple(k * state - 1 for state in range(1, n)) - return string_from_flags(flags, n=n, k=k) + return icdfa_string_from_flags(flags, n=n, k=k) def last_icdfa_empty_string(*, n: int, k: int) -> tuple[int, ...]: @@ -132,7 +132,7 @@ def last_icdfa_empty_string(*, n: int, k: int) -> tuple[int, ...]: if n == 1: return (0,) * k flags = list(range(n - 1)) - transitions = list(string_from_flags(flags, n=n, k=k)) + transitions = list(icdfa_string_from_flags(flags, n=n, k=k)) flag_set = set(flags) for index in range(k * n): if index in flag_set: @@ -141,7 +141,7 @@ def last_icdfa_empty_string(*, n: int, k: int) -> tuple[int, ...]: return tuple(transitions) -def next_flags(flags: list[int], *, k: int) -> None: +def icdfa_next_flags(flags: list[int], *, k: int) -> None: """Advance ``flags`` in-place to the next valid flag sequence, or raise ``StopIteration``.""" def nextflags(index: int) -> None: @@ -233,7 +233,7 @@ def nexticdfa(state: int, symbol: int) -> None: nexticdfa(n - 1, k - 1) -def count_flag_sequences(k: int, n: int) -> int: +def icdfa_count_flag_sequences(k: int, n: int) -> int: """Return ``F_{k,n}``, the number of valid flag sequences (Fuss--Catalan).""" if n <= 1: return 1 @@ -284,17 +284,17 @@ def iter_icdfa_empty_strings(k: int, n: int) -> Iterator[tuple[int, ...]]: return flags = [k * state - 1 for state in range(1, n)] - transitions = list(string_from_flags(flags, n=n, k=k)) + transitions = list(icdfa_string_from_flags(flags, n=n, k=k)) while True: yield tuple(transitions) try: next_icdfa_empty_string(transitions, flags, n=n, k=k) except StopIteration: try: - next_flags(flags, k=k) + icdfa_next_flags(flags, k=k) except StopIteration: break - transitions[:] = list(string_from_flags(flags, n=n, k=k)) + transitions[:] = list(icdfa_string_from_flags(flags, n=n, k=k)) def iter_icdfa(k: int, n: int) -> Iterator[tuple[tuple[int, ...], frozenset[int]]]: diff --git a/sofic/automata/idfa.py b/sofic/automata/enumeration/idfa.py similarity index 94% rename from sofic/automata/idfa.py rename to sofic/automata/enumeration/idfa.py index ad7db6d..72f9357 100644 --- a/sofic/automata/idfa.py +++ b/sofic/automata/enumeration/idfa.py @@ -11,10 +11,10 @@ from collections.abc import Iterator, Sequence from functools import cache -from sofic.automata.icdfa import ( +from sofic.automata.enumeration.icdfa import ( _upper_bound_at, _validate_flags, - next_flags, + icdfa_next_flags, ) from sofic.exceptions import SoficValidationError @@ -22,14 +22,14 @@ "MISSING_TRANSITION", "count_accessible_idfa", "count_idfa_strings_for_flags", - "extended_flags", + "idfa_extended_flags", "first_idfa_string", "idfa_string_to_topological_graph", "iter_idfa_strings", "last_idfa_string", "next_idfa_string", "rank_idfa_string", - "transition_count", + "idfa_transition_count", "unrank_idfa_string", "validate_idfa_string", ] @@ -41,7 +41,7 @@ class IDFAEnumerationError(SoficValidationError): """Raised when incomplete accessible DFA enumeration fails.""" -def extended_flags(flags: Sequence[int], *, n: int, k: int) -> tuple[int, ...]: +def idfa_extended_flags(flags: Sequence[int], *, n: int, k: int) -> tuple[int, ...]: """Return ``(f_0, …, f_n)`` with ``f_0 = -1`` and ``f_n = nk``.""" return (MISSING_TRANSITION,) + tuple(flags) + (n * k,) @@ -71,12 +71,12 @@ def validate_idfa_string( raise IDFAEnumerationError(f"state {state} does not appear in the first {k * state} symbols") -def transition_count(transitions: Sequence[int]) -> int: +def idfa_transition_count(transitions: Sequence[int]) -> int: """Return the number of defined transitions in ``transitions``.""" return sum(1 for value in transitions if value != MISSING_TRANSITION) -def string_from_flags( +def idfa_string_from_flags( flags: Sequence[int], *, n: int, @@ -98,7 +98,7 @@ def first_idfa_string(*, n: int, k: int) -> tuple[int, ...]: if n == 1: return (MISSING_TRANSITION,) * k flags = tuple(k * state - 1 for state in range(1, n)) - return string_from_flags(flags, n=n, k=k) + return idfa_string_from_flags(flags, n=n, k=k) def last_idfa_string(*, n: int, k: int) -> tuple[int, ...]: @@ -106,7 +106,7 @@ def last_idfa_string(*, n: int, k: int) -> tuple[int, ...]: if n == 1: return (n - 1,) * k flags = list(range(n - 1)) - transitions = list(string_from_flags(flags, n=n, k=k)) + transitions = list(idfa_string_from_flags(flags, n=n, k=k)) flag_set = set(flags) for index in range(k * n): if index in flag_set: @@ -185,7 +185,7 @@ def nextidfa(state: int, symbol: int) -> None: def count_idfa_strings_for_flags(flags: Sequence[int], *, n: int, k: int) -> int: """Return the number of IDFA∅ strings with the given flag sequence.""" - ext = extended_flags(flags, n=n, k=k) + ext = idfa_extended_flags(flags, n=n, k=k) product = 1 for segment_index in range(n): segment = ext[segment_index + 1] - ext[segment_index] - 1 @@ -261,17 +261,17 @@ def _iter_idfa_strings_impl(k: int, n: int) -> Iterator[tuple[int, ...]]: return flags = [k * state - 1 for state in range(1, n)] - transitions = list(string_from_flags(flags, n=n, k=k)) + transitions = list(idfa_string_from_flags(flags, n=n, k=k)) while True: yield tuple(transitions) try: next_idfa_string(transitions, flags, n=n, k=k) except StopIteration: try: - next_flags(flags, k=k) + icdfa_next_flags(flags, k=k) except StopIteration: break - transitions[:] = list(string_from_flags(flags, n=n, k=k)) + transitions[:] = list(idfa_string_from_flags(flags, n=n, k=k)) def iter_idfa_strings(k: int, n: int) -> Iterator[tuple[int, ...]]: diff --git a/sofic/automata/enumeration.py b/sofic/automata/enumeration/words.py similarity index 86% rename from sofic/automata/enumeration.py rename to sofic/automata/enumeration/words.py index 3da18f2..c5353f7 100644 --- a/sofic/automata/enumeration.py +++ b/sofic/automata/enumeration/words.py @@ -6,6 +6,7 @@ from itertools import product from typing import Any +from sofic.automata.algorithms import _effective_alphabet from sofic.automata.base import LabeledAutomaton @@ -36,9 +37,3 @@ def iter_language( while max_length is None or length <= max_length: yield from words_of_length(automaton, length) length += 1 - - -def _effective_alphabet(automaton: LabeledAutomaton) -> frozenset[Any]: - from sofic.automata.algorithms import _effective_alphabet as _shared - - return _shared(automaton) diff --git a/sofic/automata/languages/__init__.py b/sofic/automata/languages/__init__.py index 5f3224b..6756b45 100644 --- a/sofic/automata/languages/__init__.py +++ b/sofic/automata/languages/__init__.py @@ -1,6 +1,6 @@ """Regular-language algebra for automata constructions.""" -from sofic.automata.languages.atoms import atoms, is_prime_atom, prime_atoms +from sofic.automata.languages.atoms import atoms, prime_atoms from sofic.automata.languages.base import AutomatonLanguage, ExplicitLanguage, RegularLanguage from sofic.automata.languages.operations import ( complement, @@ -25,7 +25,6 @@ "difference", "intersection", "is_composed_residual", - "is_prime_atom", "kleene_star", "left_quotient", "left_quotients", diff --git a/sofic/automata/languages/_quotient_utils.py b/sofic/automata/languages/_quotient_utils.py index 8adceb7..e4f524d 100644 --- a/sofic/automata/languages/_quotient_utils.py +++ b/sofic/automata/languages/_quotient_utils.py @@ -6,24 +6,7 @@ from typing import Any from sofic.automata.languages.automaton_ops import minimal_dfa_from_language -from sofic.automata.languages.base import AutomatonLanguage, ExplicitLanguage, RegularLanguage, as_language - - -def _words_up_to(length: int, alphabet: frozenset[Any]) -> list[tuple[Any, ...]]: - if length < 0: - return [] - if length == 0: - return [()] - shorter = _words_up_to(length - 1, alphabet) - return [word + (symbol,) for word in shorter for symbol in alphabet] - - -def _languages_equal(left: RegularLanguage, right: RegularLanguage, alphabet: frozenset[Any], max_len: int = 6) -> bool: - for length in range(max_len + 1): - for word in _words_up_to(length, alphabet): - if (word in left) != (word in right): - return False - return True +from sofic.automata.languages.base import AutomatonLanguage, ExplicitLanguage def _residual_from_state(aut: AutomatonLanguage, state) -> AutomatonLanguage: @@ -32,15 +15,6 @@ def _residual_from_state(aut: AutomatonLanguage, state) -> AutomatonLanguage: return AutomatonLanguage(minimal_dfa_from_language(sub)) -def _alphabet_of(language: RegularLanguage) -> frozenset[Any]: - lang = as_language(language) # type: ignore[arg-type] - if isinstance(lang, ExplicitLanguage): - return lang.alphabet - if isinstance(lang, AutomatonLanguage): - return lang.automaton.input_alphabet - return frozenset() - - def _suffixes_if_prefix(word: tuple[Any, ...], prefix: Sequence[Any]) -> set[tuple[Any, ...]]: p = tuple(prefix) if word[: len(p)] == p: diff --git a/sofic/automata/languages/atoms.py b/sofic/automata/languages/atoms.py index dcdfa25..fa625fe 100644 --- a/sofic/automata/languages/atoms.py +++ b/sofic/automata/languages/atoms.py @@ -1,31 +1,41 @@ -"""Atoms and prime atoms of regular languages.""" +r"""Atoms and prime atoms of regular languages :cite:`BrzozowskiTamm2014`. + +An *atom* of ``L`` is a non-empty intersection of complemented or +uncomplemented left quotients of ``L``. The atoms correspond one-to-one with +the states ``q`` of the minimal DFA of the reversed language :math:`L^R`: the +atom :math:`A_q` is the reverse of the set of words leading to ``q``, and these +are exactly the right languages of the átomaton's states. An atom is *prime* +when its matching quotient of :math:`L^R` is a prime residual; prime atoms label +the states of the maximized prime átomaton. +""" from __future__ import annotations -from sofic.automata.languages._quotient_utils import _alphabet_of, _languages_equal, _residual_from_state -from sofic.automata.languages.automaton_ops import minimal_dfa_from_language from sofic.automata.languages.base import AutomatonLanguage, RegularLanguage, as_language -from sofic.automata.languages.quotients import left_quotients -def atoms(language: RegularLanguage) -> frozenset[RegularLanguage]: +def _reversed_table(language: RegularLanguage): + from sofic.automata.canonical.residual import ResidualTable + lang = as_language(language) # type: ignore[arg-type] - result: set[RegularLanguage] = set(left_quotients(language)) - if isinstance(lang, AutomatonLanguage) and not result: - dfa = minimal_dfa_from_language(lang.automaton) - for state in dfa.states(): - result.add(_residual_from_state(AutomatonLanguage(dfa), state)) - return frozenset(result) + if not isinstance(lang, AutomatonLanguage): + raise TypeError("atoms are defined for automaton-backed regular languages") + return ResidualTable.from_automaton(lang.automaton.reverse()) -def prime_atoms(language: RegularLanguage) -> frozenset[RegularLanguage]: - all_atoms = atoms(language) - return frozenset(atom for atom in all_atoms if is_prime_atom(atom, all_atoms)) +def _atom(table, state) -> AutomatonLanguage: + from sofic.automata.languages.automaton_ops import minimal_dfa_from_language + + return AutomatonLanguage(minimal_dfa_from_language(table.left_language_automaton(state).reverse())) -def is_prime_atom(atom: RegularLanguage, all_atoms: frozenset[RegularLanguage]) -> bool: - others = [candidate for candidate in all_atoms if candidate is not atom] - if not others: - return True - alphabet = _alphabet_of(atom) - return all(not _languages_equal(atom, other, alphabet) for other in others) +def atoms(language: RegularLanguage) -> frozenset[RegularLanguage]: + """Return the atoms of ``language``.""" + table = _reversed_table(language) + return frozenset(_atom(table, q) for q in table.states if not table.is_empty(q)) + + +def prime_atoms(language: RegularLanguage) -> frozenset[RegularLanguage]: + """Return the prime atoms of ``language``.""" + table = _reversed_table(language) + return frozenset(_atom(table, q) for q in table.prime_states()) diff --git a/sofic/automata/languages/automaton_ops.py b/sofic/automata/languages/automaton_ops.py index 593e472..c119d57 100644 --- a/sofic/automata/languages/automaton_ops.py +++ b/sofic/automata/languages/automaton_ops.py @@ -6,8 +6,8 @@ from collections.abc import Callable, Hashable, Sequence from typing import Any -from sofic.automata.algorithms import complete, determinize, minimize, trim -from sofic.automata.base import LabeledAutomaton +from sofic.automata.algorithms import _effective_alphabet, complete, determinize, minimize, trim +from sofic.automata.base import LabeledAutomaton, run_nfa from sofic.automata.dfa import DFA from sofic.automata.nfa import NFA from sofic.graph import ATTR_SYMBOL, EPSILON, TransitionGraph @@ -202,12 +202,7 @@ def kleene_star_nfa(aut: LabeledAutomaton) -> NFA: def left_quotient_automaton(u: Sequence[Any], aut: LabeledAutomaton) -> NFA: nfa = _to_nfa(aut) - current = nfa.epsilon_closure(set(nfa.initial_states)) - for symbol in u: - next_states: set[Hashable] = set() - for state in current: - next_states.update(nfa.delta(state, symbol)) - current = nfa.epsilon_closure(next_states) + current = run_nfa(nfa, u) return NFA( input_alphabet=nfa.input_alphabet, initial_states=frozenset(current), @@ -235,9 +230,3 @@ def state_residual_languages(dfa: DFA) -> dict[Hashable, DFA]: sub.accepting_states = dfa.accepting_states residuals[state] = minimize(_to_dfa(sub)) return residuals - - -def _effective_alphabet(aut: LabeledAutomaton) -> frozenset[Any]: - from sofic.automata.algorithms import _effective_alphabet as _shared - - return _shared(aut) diff --git a/sofic/automata/languages/quotients.py b/sofic/automata/languages/quotients.py index 070cb16..ad32484 100644 --- a/sofic/automata/languages/quotients.py +++ b/sofic/automata/languages/quotients.py @@ -5,11 +5,11 @@ from collections.abc import Sequence from typing import Any +from sofic.automata._words import _words_up_to from sofic.automata.languages._quotient_utils import ( _prefixes_if_suffix, _residual_from_state, _suffixes_if_prefix, - _words_up_to, ) from sofic.automata.languages.automaton_ops import ( left_quotient_automaton, @@ -54,9 +54,8 @@ def left_quotients(language: RegularLanguage) -> frozenset[RegularLanguage]: if isinstance(lang, ExplicitLanguage): alphabet = lang.alphabet quotients: set[RegularLanguage] = set() - for length in range(4): - for word in _words_up_to(length, alphabet): - quotients.add(left_quotient(word, lang)) + for word in _words_up_to(3, alphabet): + quotients.add(left_quotient(word, lang)) return frozenset(quotients) raise TypeError(f"unsupported language type {type(lang)!r}") diff --git a/sofic/automata/languages/residuals.py b/sofic/automata/languages/residuals.py index ddaee4f..c412f08 100644 --- a/sofic/automata/languages/residuals.py +++ b/sofic/automata/languages/residuals.py @@ -1,25 +1,65 @@ -"""Prime and composed residual languages.""" +"""Prime and composed residual languages :cite:`Denis2002`. + +A residual (left quotient) of ``L`` is *composed* when it is the union of the +residuals strictly contained in it, and *prime* otherwise (the empty residual +is the empty union, so it is composed). For automaton-backed languages these +questions are decided exactly on the minimal DFA; for an +:class:`~sofic.automata.languages.base.ExplicitLanguage` (a finite labeled +sample) they are answered on the sample only. +""" from __future__ import annotations -from sofic.automata.languages._quotient_utils import ( - _alphabet_of, - _is_union_of_others, - _languages_equal, -) -from sofic.automata.languages.base import ExplicitLanguage, RegularLanguage +from sofic.automata.languages._quotient_utils import _is_union_of_others +from sofic.automata.languages.base import AutomatonLanguage, ExplicitLanguage, RegularLanguage, as_language from sofic.automata.languages.quotients import left_quotients def prime_residuals(language: RegularLanguage) -> frozenset[RegularLanguage]: - all_residuals = left_quotients(language) + """Return the prime residuals of ``language``.""" + lang = as_language(language) # type: ignore[arg-type] + if isinstance(lang, AutomatonLanguage): + from sofic.automata.canonical.residual import ResidualTable + + table = ResidualTable.from_automaton(lang.automaton) + return frozenset(AutomatonLanguage(table.residual_automaton(q)) for q in table.prime_states()) + all_residuals = left_quotients(lang) return frozenset(r for r in all_residuals if not is_composed_residual(r, all_residuals)) def is_composed_residual(residual: RegularLanguage, all_residuals: frozenset[RegularLanguage]) -> bool: + """Return whether ``residual`` is the union of the members of ``all_residuals`` strictly inside it.""" if isinstance(residual, ExplicitLanguage): others = [r for r in all_residuals if r is not residual and isinstance(r, ExplicitLanguage)] return _is_union_of_others(residual, others) - others = [r for r in all_residuals if r is not residual] - alphabet = _alphabet_of(residual) - return any(_languages_equal(residual, other, alphabet) for other in others) + + from sofic.automata.languages.automaton_ops import union_nfa + + target = as_language(residual).automaton # type: ignore[union-attr] + inside = [] + for other in all_residuals: + if other is residual: + continue + candidate = as_language(other).automaton # type: ignore[union-attr] + if _is_subset(candidate, target) and not _is_subset(target, candidate): + inside.append(candidate) + if not inside: + return _is_empty(target) + union = inside[0] + for candidate in inside[1:]: + union = union_nfa(union, candidate) + return _is_subset(target, union) + + +def _is_subset(smaller, larger) -> bool: + from sofic.automata.algorithms import _transition_alphabet + from sofic.automata.languages.automaton_ops import difference_dfa + + alphabet = _transition_alphabet(smaller) | _transition_alphabet(larger) + return _is_empty(difference_dfa(smaller, larger, alphabet=alphabet)) + + +def _is_empty(aut) -> bool: + from sofic.automata.algorithms import trim + + return not trim(aut).accepting_states diff --git a/sofic/automata/learning.py b/sofic/automata/learning.py deleted file mode 100644 index 021cc07..0000000 --- a/sofic/automata/learning.py +++ /dev/null @@ -1,79 +0,0 @@ -"""NL* active learning for maximized prime átomata.""" - -from __future__ import annotations - -from collections.abc import Sequence -from typing import Any - -from sofic.automata.atomaton import MaximizedPrimeAtomaton -from sofic.automata.languages.base import RegularLanguage -from sofic.automata.observation import ObservationTable - - -def learn_maximized_prime_atomaton( - teacher: RegularLanguage, - alphabet: frozenset[Any], - max_rounds: int = 32, -) -> MaximizedPrimeAtomaton: - """Simulated NL* teacher loop using membership queries.""" - access_words: set[tuple[Any, ...]] = {()} - experiments: set[tuple[Any, ...]] = {()} - - def membership(word: Sequence[Any]) -> bool: - return tuple(word) in teacher - - def close_table(max_depth: int = 6) -> None: - changed = True - while changed: - changed = False - for word in list(access_words): - if len(word) >= max_depth: - continue - for symbol in alphabet: - successor = word + (symbol,) - if successor not in access_words: - access_words.add(successor) - changed = True - - for _ in range(max_rounds): - close_table() - table = _build_table(access_words, experiments, membership) - hypothesis = table.to_maximized_prime_atomaton() - counterexample = _find_counterexample(teacher, hypothesis, alphabet) - if counterexample is None: - return hypothesis - for length in range(len(counterexample) + 1): - experiments.add(counterexample[length:]) - - return _build_table(access_words, experiments, membership).to_maximized_prime_atomaton() - - -def _build_table( - access_words: set[tuple[Any, ...]], - experiments: set[tuple[Any, ...]], - membership, -) -> ObservationTable: - membership_map: dict[tuple[Any, ...], bool] = {} - for prefix in access_words: - for suffix in experiments: - membership_map[prefix + suffix] = membership(prefix + suffix) - return ObservationTable( - access_words=frozenset(access_words), - experiments=frozenset(experiments), - membership=membership_map, - ) - - -def _find_counterexample( - teacher: RegularLanguage, - hypothesis: MaximizedPrimeAtomaton, - alphabet: frozenset[Any], - max_len: int = 8, -) -> tuple[Any, ...] | None: - from sofic.automata.languages._quotient_utils import _words_up_to - - for length in range(max_len + 1): - for word in _words_up_to(length, alphabet): - if (word in teacher) != hypothesis.recognizes(word): - return word - return None diff --git a/sofic/automata/learning/__init__.py b/sofic/automata/learning/__init__.py new file mode 100644 index 0000000..88d9c5d --- /dev/null +++ b/sofic/automata/learning/__init__.py @@ -0,0 +1,69 @@ +"""Passive and active automaton learning algorithms.""" + +from sofic.automata.learning.active import ( + AutomatonEquivalenceOracle, + EquivalenceOracle, + ExhaustiveEquivalenceOracle, + FunctionMealyOracle, + FunctionMembershipOracle, + LanguageMembershipOracle, + MealyEquivalenceOracle, + MealyExhaustiveEquivalenceOracle, + MealyMembershipOracle, + MembershipOracle, + RandomWalkEquivalenceOracle, + TransducerOutputOracle, + learn_dfa_from_language, + learn_dfa_lstar, + learn_dfa_ttt, + learn_mealy_from_transducer, + learn_mealy_lstar, +) +from sofic.automata.learning.alergia import learn_pfa_alergia +from sofic.automata.learning.dfasat import learn_dfa_sat +from sofic.automata.learning.edsm import learn_dfa_edsm +from sofic.automata.learning.nlstar import learn_prime_atomaton_nlstar, learn_rfsa_from_language, learn_rfsa_nlstar +from sofic.automata.learning.observation import ObservationTable +from sofic.automata.learning.papni import ( + DyckAlphabet, + encode_dyck_samples, + encode_dyck_word, + is_well_matched, + learn_sofic_dyck_shift_papni, + sofic_dyck_shift_from_papni_dfa, +) +from sofic.automata.learning.rpni import learn_dfa_rpni + +__all__ = [ + "AutomatonEquivalenceOracle", + "DyckAlphabet", + "EquivalenceOracle", + "ExhaustiveEquivalenceOracle", + "FunctionMealyOracle", + "FunctionMembershipOracle", + "LanguageMembershipOracle", + "MealyEquivalenceOracle", + "MealyExhaustiveEquivalenceOracle", + "MealyMembershipOracle", + "MembershipOracle", + "ObservationTable", + "RandomWalkEquivalenceOracle", + "TransducerOutputOracle", + "is_well_matched", + "learn_dfa_edsm", + "learn_dfa_from_language", + "learn_dfa_lstar", + "learn_dfa_rpni", + "learn_dfa_sat", + "learn_dfa_ttt", + "learn_mealy_from_transducer", + "learn_mealy_lstar", + "learn_pfa_alergia", + "learn_prime_atomaton_nlstar", + "learn_rfsa_from_language", + "learn_rfsa_nlstar", + "learn_sofic_dyck_shift_papni", + "encode_dyck_word", + "encode_dyck_samples", + "sofic_dyck_shift_from_papni_dfa", +] diff --git a/sofic/automata/active.py b/sofic/automata/learning/active.py similarity index 82% rename from sofic/automata/active.py rename to sofic/automata/learning/active.py index df47fd0..386dd71 100644 --- a/sofic/automata/active.py +++ b/sofic/automata/learning/active.py @@ -13,8 +13,8 @@ * a redundancy-free **discrimination-tree** learner in the TTT family :cite:`KearnsVazirani1994,Isberner2014`. -These complement the existing NL\* átomaton learner -(:func:`sofic.automata.learning.learn_maximized_prime_atomaton`). +These complement the NL\* canonical-RFSA learner +(:func:`sofic.automata.learning.nlstar.learn_rfsa_nlstar`). """ from __future__ import annotations @@ -24,6 +24,7 @@ import numpy as np +from sofic.automata._words import _words_up_to from sofic.automata.dfa import DFA from sofic.automata.transducers import MealyMachine @@ -36,6 +37,7 @@ "LanguageMembershipOracle", "FunctionMealyOracle", "TransducerOutputOracle", + "AutomatonEquivalenceOracle", "ExhaustiveEquivalenceOracle", "RandomWalkEquivalenceOracle", "MealyExhaustiveEquivalenceOracle", @@ -134,35 +136,96 @@ def output(self, word: Sequence[Any]) -> Word: return next(iter(outputs)) -def _words_up_to(max_length: int, alphabet: Sequence[Any]) -> Iterator[Word]: - frontier: list[Word] = [()] - yield () - for _ in range(max_length): - nxt: list[Word] = [] - for word in frontier: - for symbol in alphabet: - extended = (*word, symbol) - yield extended - nxt.append(extended) - frontier = nxt +class _BoundedEquivalenceOracle: + """Approximate equivalence test over a finite stream of candidate words. + + Subclasses supply the candidate words (:meth:`_words`) and the comparison of + target versus hypothesis on one word (:meth:`_disagrees`); the first + disagreeing word is the counterexample. + """ + + def __init__(self, alphabet: Iterable[Any]) -> None: + self._alphabet = tuple(sorted(alphabet, key=repr)) + + def _words(self) -> Iterator[Word]: + raise NotImplementedError + + def _disagrees(self, hypothesis: Any, word: Word) -> bool: + raise NotImplementedError + + def find_counterexample(self, hypothesis: Any) -> Word | None: + for word in self._words(): + if self._disagrees(hypothesis, word): + return word + return None -class ExhaustiveEquivalenceOracle: +class _MembershipBoundedOracle(_BoundedEquivalenceOracle): + """Bounded oracle comparing a membership oracle against a DFA hypothesis.""" + + def __init__(self, membership: MembershipOracle, alphabet: Iterable[Any]) -> None: + super().__init__(alphabet) + self._membership = membership + + def _disagrees(self, hypothesis: DFA, word: Word) -> bool: + return self._membership.member(word) != hypothesis.recognizes(word) + + +class ExhaustiveEquivalenceOracle(_MembershipBoundedOracle): """Bounded exhaustive equivalence test for DFA hypotheses.""" def __init__(self, membership: MembershipOracle, alphabet: Iterable[Any], *, max_length: int = 10) -> None: - self._membership = membership - self._alphabet = tuple(sorted(alphabet, key=repr)) + super().__init__(membership, alphabet) self._max_length = int(max_length) - def find_counterexample(self, hypothesis: DFA) -> Word | None: - for word in _words_up_to(self._max_length, self._alphabet): - if self._membership.member(word) != hypothesis.recognizes(word): + def _words(self) -> Iterator[Word]: + return _words_up_to(self._max_length, self._alphabet) + + +class AutomatonEquivalenceOracle: + """Exact equivalence against a target finite automaton. + + Returns a shortest word on which the hypothesis and the target disagree, + found by breadth-first search over the product of their subset + constructions, or ``None`` when they recognize the same language. + """ + + def __init__(self, target: Any, alphabet: Iterable[Any] | None = None) -> None: + self._target = target + self._alphabet = None if alphabet is None else frozenset(alphabet) + + def find_counterexample(self, hypothesis: Any) -> Word | None: + from sofic.automata.algorithms import _transition_alphabet + + symbols = (self._alphabet or frozenset()) | _transition_alphabet(self._target) + symbols |= _transition_alphabet(hypothesis) + ordered = tuple(sorted(symbols, key=repr)) + start = (_closure(hypothesis, hypothesis.initial_states), _closure(self._target, self._target.initial_states)) + seen = {start} + queue: list[tuple[tuple[frozenset[Any], frozenset[Any]], Word]] = [(start, ())] + for (left, right), word in queue: + if bool(left & hypothesis.accepting_states) != bool(right & self._target.accepting_states): return word + for symbol in ordered: + successor = (_step(hypothesis, left, symbol), _step(self._target, right, symbol)) + if successor not in seen: + seen.add(successor) + queue.append((successor, (*word, symbol))) return None -class RandomWalkEquivalenceOracle: +def _closure(aut: Any, states: Iterable[Any]) -> frozenset[Any]: + return frozenset(aut.epsilon_closure(set(states))) + + +def _step(aut: Any, states: frozenset[Any], symbol: Any) -> frozenset[Any]: + targets: set[Any] = set() + for state in states: + targets |= aut.delta(state, symbol) + return _closure(aut, targets) + + +class RandomWalkEquivalenceOracle(_MembershipBoundedOracle): """Randomized equivalence test drawing random input words for DFA hypotheses.""" def __init__( @@ -174,39 +237,33 @@ def __init__( max_steps: int = 30, rng: np.random.Generator | int | None = None, ) -> None: - self._membership = membership - self._alphabet = tuple(sorted(alphabet, key=repr)) + super().__init__(membership, alphabet) self._num_walks = int(num_walks) self._max_steps = int(max_steps) self._rng = rng if isinstance(rng, np.random.Generator) else np.random.default_rng(rng) - def find_counterexample(self, hypothesis: DFA) -> Word | None: + def _words(self) -> Iterator[Word]: n_symbols = len(self._alphabet) for _ in range(self._num_walks): length = int(self._rng.integers(0, self._max_steps + 1)) - word = tuple(self._alphabet[int(self._rng.integers(0, n_symbols))] for _ in range(length)) - if self._membership.member(word) != hypothesis.recognizes(word): - return word - return None + yield tuple(self._alphabet[int(self._rng.integers(0, n_symbols))] for _ in range(length)) -class MealyExhaustiveEquivalenceOracle: +class MealyExhaustiveEquivalenceOracle(_BoundedEquivalenceOracle): """Bounded exhaustive equivalence test for Mealy hypotheses.""" def __init__(self, oracle: MealyMembershipOracle, alphabet: Iterable[Any], *, max_length: int = 10) -> None: + super().__init__(alphabet) self._oracle = oracle - self._alphabet = tuple(sorted(alphabet, key=repr)) self._max_length = int(max_length) - def find_counterexample(self, hypothesis: MealyMachine) -> Word | None: - for word in _words_up_to(self._max_length, self._alphabet): - if not word: - continue - produced = hypothesis.transduce(word) - hyp_out = next(iter(produced)) if produced else None - if self._oracle.output(word) != hyp_out: - return word - return None + def _words(self) -> Iterator[Word]: + return (word for word in _words_up_to(self._max_length, self._alphabet) if word) + + def _disagrees(self, hypothesis: MealyMachine, word: Word) -> bool: + produced = hypothesis.transduce(word) + hyp_out = next(iter(produced)) if produced else None + return self._oracle.output(word) != hyp_out # ------------------------------------------------------------------------------ L* @@ -408,10 +465,6 @@ def build() -> DFA: raise RuntimeError("TTT did not converge within max_rounds; check the equivalence oracle") -def _hypothesis_access(word: Word, sift: Callable[[Word], _DTNode]) -> Word: - return sift(word).access - - def _split_leaf( counterexample: Word, sift: Callable[[Word], _DTNode], @@ -422,7 +475,7 @@ def _split_leaf( length = len(counterexample) def alpha(index: int) -> Word: - return _hypothesis_access(counterexample[:index], sift) + counterexample[index:] + return sift(counterexample[:index]).access + counterexample[index:] base = member(alpha(0)) breakpoint_index = None @@ -433,7 +486,7 @@ def alpha(index: int) -> Word: if breakpoint_index is None: # pragma: no cover - guaranteed by a valid counterexample raise RuntimeError("counterexample analysis found no breakpoint") - state_access = _hypothesis_access(counterexample[:breakpoint_index], sift) + state_access = sift(counterexample[:breakpoint_index]).access symbol = counterexample[breakpoint_index] discriminator = counterexample[breakpoint_index + 1 :] new_access = state_access + (symbol,) diff --git a/sofic/automata/alergia.py b/sofic/automata/learning/alergia.py similarity index 99% rename from sofic/automata/alergia.py rename to sofic/automata/learning/alergia.py index 1df3f26..2adc9a0 100644 --- a/sofic/automata/alergia.py +++ b/sofic/automata/learning/alergia.py @@ -5,7 +5,7 @@ whenever a Hoeffding-bound test cannot distinguish their outgoing (and recursive) transition statistics. It is the stochastic, unlabeled counterpart of RPNI/EDSM and a state-merging alternative to Causal-State Splitting Reconstruction -(:func:`sofic.generators.epsilon_inference.cssr`). +(:func:`sofic.inference.cssr.process.cssr`). The learned automaton is returned as a :class:`~sofic.generators.pfa.ProbabilisticFiniteAutomaton` describing the diff --git a/sofic/automata/dfasat.py b/sofic/automata/learning/dfasat.py similarity index 95% rename from sofic/automata/dfasat.py rename to sofic/automata/learning/dfasat.py index a3cdac5..cbe6858 100644 --- a/sofic/automata/dfasat.py +++ b/sofic/automata/learning/dfasat.py @@ -7,7 +7,7 @@ into a graph-colouring SAT instance -- one Boolean per (APTA node, colour), plus accepting-colour and transition variables -- and an external SAT solver is asked whether a ``k``-colouring exists. The search runs ``k`` upward from a lower bound -to the EDSM upper bound (:func:`sofic.automata.edsm.learn_dfa_edsm`); the first +to the EDSM upper bound (:func:`sofic.automata.learning.edsm.learn_dfa_edsm`); the first satisfiable ``k`` is the provably minimal DFA. This requires the optional `python-sat `_ dependency @@ -20,7 +20,7 @@ from typing import Any from sofic.automata.dfa import DFA -from sofic.automata.edsm import _ACCEPT, _REJECT, _build_apta, _collect_alphabet, learn_dfa_edsm +from sofic.automata.learning.edsm import _ACCEPT, _REJECT, _build_apta, _collect_alphabet, learn_dfa_edsm __all__ = ["learn_dfa_sat"] @@ -156,7 +156,7 @@ def learn_dfa_sat( Smallest state count to try (default ``1``). upper_bound Largest state count to try. Defaults to the number of states of the - EDSM hypothesis (:func:`sofic.automata.edsm.learn_dfa_edsm`), which is a + EDSM hypothesis (:func:`sofic.automata.learning.edsm.learn_dfa_edsm`), which is a valid upper bound on the minimum. solver_name Any `python-sat `_ solver name (default diff --git a/sofic/automata/edsm.py b/sofic/automata/learning/edsm.py similarity index 99% rename from sofic/automata/edsm.py rename to sofic/automata/learning/edsm.py index e2b8e32..8c46c5f 100644 --- a/sofic/automata/edsm.py +++ b/sofic/automata/learning/edsm.py @@ -1,6 +1,6 @@ """Passive DFA learning via Evidence-Driven State Merging (blue-fringe). -EDSM upgrades the greedy RPNI merge order (:func:`sofic.automata.rpni.learn_dfa_rpni`) +EDSM upgrades the greedy RPNI merge order (:func:`sofic.automata.learning.rpni.learn_dfa_rpni`) with the evidence-driven, red/blue "blue-fringe" strategy that won the Abbadingo One competition :cite:`Lang1998`. Starting from the augmented prefix-tree acceptor of the labeled sample, it maintains a set of confirmed *red* states and diff --git a/sofic/automata/learning/nlstar.py b/sofic/automata/learning/nlstar.py new file mode 100644 index 0000000..a73114a --- /dev/null +++ b/sofic/automata/learning/nlstar.py @@ -0,0 +1,202 @@ +r"""NL\*: Angluin-style active learning of canonical RFSAs :cite:`Bollig2009`. + +NL\* keeps an observation table with rows for the access words ``U`` and their +one-symbol extensions ``U Sigma``, and columns for a suffix-closed set ``V`` of +experiments. Rows are compared pointwise: ``r <= r'`` when ``r'`` accepts every +experiment ``r`` accepts, and the join of rows is their pointwise ``or``. A row +is *prime* when it is not the join of the rows strictly below it. + +The table is + +* **RFSA-closed** when every row of ``U Sigma`` is the join of the prime rows of + ``U`` below it, and +* **RFSA-consistent** when ``row(u') <= row(u)`` implies + ``row(u' a) <= row(u a)`` for all ``u, u'`` in ``U`` and symbols ``a``. + +A closed, consistent table yields the hypothesis whose states are the prime +rows of ``U``. Counterexamples add all of their suffixes to ``V``. When the +equivalence oracle accepts, the hypothesis is the canonical RFSA of the target. +""" + +from __future__ import annotations + +from collections.abc import Iterable, Sequence +from typing import Any + +from sofic.automata.base import LabeledAutomaton +from sofic.automata.canonical.atomaton import MaximizedPrimeAtomaton +from sofic.automata.canonical.rfsa import CanonicalRFSA +from sofic.automata.learning.active import ( + AutomatonEquivalenceOracle, + EquivalenceOracle, + ExhaustiveEquivalenceOracle, + LanguageMembershipOracle, + MembershipOracle, + _MembershipCache, +) + +Word = tuple[Any, ...] +Row = tuple[bool, ...] + + +def _leq(left: Row, right: Row) -> bool: + return all(not a or b for a, b in zip(left, right, strict=True)) + + +def _join(rows: Iterable[Row], width: int) -> Row: + result = [False] * width + for row in rows: + result = [a or b for a, b in zip(result, row, strict=True)] + return tuple(result) + + +def _primes(rows: set[Row], width: int) -> set[Row]: + return {row for row in rows if _join((r for r in rows if r != row and _leq(r, row)), width) != row} + + +def learn_rfsa_nlstar( + alphabet: Iterable[Any], + membership: MembershipOracle, + equivalence: EquivalenceOracle, + *, + max_rounds: int = 100, +) -> CanonicalRFSA: + r"""Learn the canonical RFSA of the target language with NL\* :cite:`Bollig2009`.""" + symbols = tuple(sorted(alphabet, key=repr)) + member = _MembershipCache(membership).member + access: list[Word] = [()] + experiments: list[Word] = [()] + + def row(word: Word) -> Row: + return tuple(member(word + suffix) for suffix in experiments) + + for _ in range(max_rounds): + while True: + width = len(experiments) + upper = {u: row(u) for u in access} + lower = {u + (a,): row(u + (a,)) for u in access for a in symbols} + all_primes = _primes(set(upper.values()) | set(lower.values()), width) + primes_upper = all_primes & set(upper.values()) + + unclosed = next( + ( + w + for w, r in sorted(lower.items(), key=lambda item: (len(item[0]), repr(item[0]))) + if r in all_primes and r not in upper.values() + ), + None, + ) + if unclosed is not None: + access.append(unclosed) + continue + + inconsistency = _find_inconsistency(access, symbols, upper, row, experiments) + if inconsistency is not None: + experiments.append(inconsistency) + continue + break + + hypothesis = _hypothesis(access, symbols, upper, primes_upper, row) + counterexample = equivalence.find_counterexample(hypothesis) + if counterexample is None: + return hypothesis + for start in range(len(counterexample) + 1): + suffix = tuple(counterexample[start:]) + if suffix not in experiments: + experiments.append(suffix) + raise RuntimeError(f"NL* did not converge within {max_rounds} equivalence rounds") + + +def _find_inconsistency(access, symbols, upper, row, experiments) -> Word | None: + for u in access: + for other in access: + if u == other or not _leq(upper[other], upper[u]): + continue + for symbol in symbols: + below, above = row(other + (symbol,)), row(u + (symbol,)) + if not _leq(below, above): + index = next(i for i, (b, a) in enumerate(zip(below, above, strict=True)) if b and not a) + return (symbol, *experiments[index]) + return None + + +def _hypothesis(access, symbols, upper, primes_upper, row) -> CanonicalRFSA: + representative: dict[Row, Word] = {} + for u in sorted(access, key=lambda w: (len(w), repr(w))): + if upper[u] in primes_upper: + representative.setdefault(upper[u], u) + states = sorted(representative, key=lambda r: (len(representative[r]), repr(representative[r]))) + name = {r: index for index, r in enumerate(states)} + epsilon_row = upper[()] + rfsa = CanonicalRFSA( + input_alphabet=frozenset(symbols), + initial_states=frozenset(name[r] for r in states if _leq(r, epsilon_row)), + accepting_states=frozenset(name[r] for r in states if r[0]), + ) + for r in states: + rfsa.graph.add_state(name[r]) + for r in states: + for symbol in symbols: + successor = row(representative[r] + (symbol,)) + for target in states: + if _leq(target, successor): + rfsa.add_transition(name[r], name[target], symbol) + return rfsa + + +class _ReversedMembership: + def __init__(self, membership: MembershipOracle) -> None: + self._membership = membership + + def member(self, word: Sequence[Any]) -> bool: + return bool(self._membership.member(tuple(reversed(tuple(word))))) + + +class _ReversedEquivalence: + def __init__(self, equivalence: EquivalenceOracle) -> None: + self._equivalence = equivalence + + def find_counterexample(self, hypothesis: Any) -> Word | None: + counterexample = self._equivalence.find_counterexample(hypothesis.reverse()) + return None if counterexample is None else tuple(reversed(tuple(counterexample))) + + +def learn_prime_atomaton_nlstar( + alphabet: Iterable[Any], + membership: MembershipOracle, + equivalence: EquivalenceOracle, + *, + max_rounds: int = 100, +) -> MaximizedPrimeAtomaton: + r"""Learn the maximized prime átomaton by running NL\* on the reversed target. + + The maximized prime átomaton of ``L`` is the reverse of the canonical RFSA of + :math:`L^R` :cite:`MaarandTamm2022`, so NL\* is run against reversed + membership and equivalence oracles and its result reversed. + """ + from sofic.automata.canonical.dual import dual_atomaton_from_rfsa + + reversed_rfsa = learn_rfsa_nlstar( + alphabet, _ReversedMembership(membership), _ReversedEquivalence(equivalence), max_rounds=max_rounds + ) + return dual_atomaton_from_rfsa(reversed_rfsa) + + +def learn_rfsa_from_language( + target: Any, + alphabet: Iterable[Any], + *, + max_length: int = 12, + max_rounds: int = 100, +) -> CanonicalRFSA: + r"""Learn the canonical RFSA of ``target`` with NL\*. + + Uses an exact equivalence oracle when ``target`` is a finite automaton and a + bounded exhaustive one (words up to ``max_length``) otherwise. + """ + membership = LanguageMembershipOracle(target) + if isinstance(target, LabeledAutomaton): + equivalence: EquivalenceOracle = AutomatonEquivalenceOracle(target, alphabet) + else: + equivalence = ExhaustiveEquivalenceOracle(membership, alphabet, max_length=max_length) + return learn_rfsa_nlstar(alphabet, membership, equivalence, max_rounds=max_rounds) diff --git a/sofic/automata/observation.py b/sofic/automata/learning/observation.py similarity index 68% rename from sofic/automata/observation.py rename to sofic/automata/learning/observation.py index 3bfd361..a0b6637 100644 --- a/sofic/automata/observation.py +++ b/sofic/automata/learning/observation.py @@ -6,9 +6,9 @@ from typing import TYPE_CHECKING, Any if TYPE_CHECKING: - from sofic.automata.atomaton import Atomaton, MaximizedPrimeAtomaton + from sofic.automata.canonical.atomaton import Atomaton, MaximizedPrimeAtomaton + from sofic.automata.canonical.rfsa import CanonicalRFSA from sofic.automata.dfa import DFA - from sofic.automata.rfsa import CanonicalRFSA @dataclass @@ -20,21 +20,21 @@ class ObservationTable: membership: dict[tuple[Any, ...], bool] = field(default_factory=dict) def to_minimal_dfa(self) -> DFA: - from sofic.automata.canonical_extraction import observation_to_minimal_dfa + from sofic.automata.canonical.residual import observation_to_minimal_dfa return observation_to_minimal_dfa(self) def to_canonical_rfsa(self) -> CanonicalRFSA: - from sofic.automata.canonical_extraction import observation_to_canonical_rfsa + from sofic.automata.canonical.residual import observation_to_canonical_rfsa return observation_to_canonical_rfsa(self) def to_atomaton(self) -> Atomaton: - from sofic.automata.canonical_extraction import observation_to_atomaton + from sofic.automata.canonical.residual import observation_to_atomaton return observation_to_atomaton(self) def to_maximized_prime_atomaton(self) -> MaximizedPrimeAtomaton: - from sofic.automata.canonical_extraction import observation_to_maximized_prime_atomaton + from sofic.automata.canonical.residual import observation_to_maximized_prime_atomaton return observation_to_maximized_prime_atomaton(self) diff --git a/sofic/automata/papni.py b/sofic/automata/learning/papni.py similarity index 95% rename from sofic/automata/papni.py rename to sofic/automata/learning/papni.py index d1c367f..c2251fc 100644 --- a/sofic/automata/papni.py +++ b/sofic/automata/learning/papni.py @@ -7,7 +7,7 @@ from typing import Any from sofic.automata.dfa import DFA -from sofic.automata.rpni import learn_dfa_rpni +from sofic.automata.learning.rpni import learn_dfa_rpni from sofic.graph import ATTR_KIND, ATTR_SYMBOL, KIND_CALL, KIND_INTERNAL, KIND_RETURN from sofic.shifts.sofic_dyck import SoficDyckShift, TransitionRef, transition_ref @@ -15,8 +15,8 @@ "DyckAlphabet", "is_well_matched", "learn_sofic_dyck_shift_papni", - "papni_encode", - "papni_encode_samples", + "encode_dyck_word", + "encode_dyck_samples", "sofic_dyck_shift_from_papni_dfa", ] @@ -63,7 +63,7 @@ def is_well_matched(word: Sequence[Any], alphabet: DyckAlphabet) -> bool: return counter == 0 -def papni_encode(word: Sequence[Any], alphabet: DyckAlphabet) -> tuple[Any, ...]: +def encode_dyck_word(word: Sequence[Any], alphabet: DyckAlphabet) -> tuple[Any, ...]: """Convert a well-matched word to its stack-aware representation (PAPNI Alg. 2).""" if not is_well_matched(word, alphabet): raise ValueError("word is not well-matched") @@ -84,7 +84,7 @@ def papni_encode(word: Sequence[Any], alphabet: DyckAlphabet) -> tuple[Any, ...] return tuple(encoded) -def papni_encode_samples( +def encode_dyck_samples( samples: Sequence[Sequence[Any]], alphabet: DyckAlphabet, *, @@ -98,7 +98,7 @@ def papni_encode_samples( if drop_non_well_matched: continue raise ValueError(f"sample {seq!r} is not well-matched") - encoded.append(papni_encode(seq, alphabet)) + encoded.append(encode_dyck_word(seq, alphabet)) return encoded @@ -218,7 +218,7 @@ def _infer_matched_edges_from_traces( for word in traces: if not is_well_matched(word, alphabet): continue - encoded = papni_encode(word, alphabet) + encoded = encode_dyck_word(word, alphabet) dfa_state = initial config_state = initial stack: list[TransitionRef] = [] @@ -280,7 +280,7 @@ def learn_sofic_dyck_shift_papni( alphabet: DyckAlphabet, ) -> SoficDyckShift: """Learn a ``SoficDyckShift`` topology from labeled samples via PAPNI + RPNI.""" - encoded_positive = papni_encode_samples(positive, alphabet) + encoded_positive = encode_dyck_samples(positive, alphabet) if not encoded_positive: raise ValueError("no well-matched positive samples remain after PAPNI filtering") @@ -290,7 +290,7 @@ def learn_sofic_dyck_shift_papni( seq = tuple(word) if not is_well_matched(seq, alphabet): continue - encoded_negative.append(papni_encode(seq, alphabet)) + encoded_negative.append(encode_dyck_word(seq, alphabet)) dfa = learn_dfa_rpni(encoded_positive, encoded_negative) well_matched_positive = [tuple(word) for word in positive if is_well_matched(word, alphabet)] diff --git a/sofic/automata/rpni.py b/sofic/automata/learning/rpni.py similarity index 100% rename from sofic/automata/rpni.py rename to sofic/automata/learning/rpni.py diff --git a/sofic/automata/nwa.py b/sofic/automata/nwa.py index 70f1ac0..94eb930 100644 --- a/sofic/automata/nwa.py +++ b/sofic/automata/nwa.py @@ -221,6 +221,46 @@ def recognizes_visible(self, symbols: Sequence[Any]) -> bool: ) return self.recognizes(word) + def union(self, other: NestedWordAutomaton) -> NestedWordAutomaton: + """Return an NWA recognizing the union with ``other``.""" + return _from_tagged_vpa(self.to_vpa().union(other.to_vpa())) + + def intersection(self, other: NestedWordAutomaton) -> NestedWordAutomaton: + """Return an NWA recognizing the intersection with ``other``.""" + return _from_tagged_vpa(self.to_vpa().intersection(other.to_vpa())) + + def complement(self) -> NestedWordAutomaton: + """Return an NWA recognizing the complement over this NWA's role alphabets.""" + return _from_tagged_vpa(self.to_vpa().complement()) + + def difference(self, other: NestedWordAutomaton) -> NestedWordAutomaton: + """Return an NWA recognizing this language minus ``other``.""" + return _from_tagged_vpa(self.to_vpa().difference(other.to_vpa())) + + def concat(self, other: NestedWordAutomaton) -> NestedWordAutomaton: + """Return an NWA recognizing concatenation with ``other``.""" + return _from_tagged_vpa(self.to_vpa().concat(other.to_vpa())) + + def kleene_star(self) -> NestedWordAutomaton: + """Return an NWA recognizing the Kleene star.""" + return _from_tagged_vpa(self.to_vpa().kleene_star()) + + def is_empty(self) -> bool: + """Return whether no nested word is accepted.""" + return self.to_vpa().is_empty() + + def is_universal(self) -> bool: + """Return whether every nested word over the role alphabets is accepted.""" + return self.to_vpa().is_universal() + + def includes(self, other: NestedWordAutomaton) -> bool: + """Return whether ``other``'s language is contained in this one.""" + return self.to_vpa().includes(other.to_vpa()) + + def equivalent(self, other: NestedWordAutomaton) -> bool: + """Return whether both NWAs recognize the same nested words.""" + return self.to_vpa().equivalent(other.to_vpa()) + @classmethod def from_vpa(cls, vpa: VisiblyPushdownAutomaton) -> NestedWordAutomaton: """Copy a visibly pushdown automaton into an equivalent NWA view.""" @@ -316,6 +356,32 @@ def to_vpa(self, *, tag_symbols: bool = True) -> VisiblyPushdownAutomaton: return result +def _from_tagged_vpa(vpa: VisiblyPushdownAutomaton) -> NestedWordAutomaton: + """Inverse of :meth:`NestedWordAutomaton.to_vpa` with role-tagged symbols.""" + result = NestedWordAutomaton( + call_alphabet=frozenset(symbol for _kind, symbol in vpa.call_alphabet), + return_alphabet=frozenset(symbol for _kind, symbol in vpa.return_alphabet), + internal_alphabet=frozenset(symbol for _kind, symbol in vpa.internal_alphabet), + hier_alphabet=vpa.stack_alphabet, + bottom_hier_state=vpa.bottom_stack_symbol, + initial_state=vpa.initial_state, + accepting_states=vpa.accepting_states, + ) + for state in vpa.states(): + result.graph.add_state(state) + for transition in vpa.transitions(): + data = transition.data + kind, (_role, symbol) = data.get(ATTR_KIND), data.get(ATTR_SYMBOL) + if kind == KIND_CALL: + result.add_call_transition(transition.source, transition.target, symbol, data.get(ATTR_STACK_SYMBOL)) + elif kind == KIND_RETURN: + for hier_state in _return_hier_states(vpa.stack_alphabet, data.get(ATTR_STACK_SYMBOL)): + result.add_return_transition(transition.source, transition.target, symbol, hier_state) + else: + result.add_internal_transition(transition.source, transition.target, symbol) + return result + + def _require_disjoint_visible_alphabets( call_alphabet: frozenset[Any], return_alphabet: frozenset[Any], diff --git a/sofic/automata/rfsa.py b/sofic/automata/rfsa.py deleted file mode 100644 index 61933c5..0000000 --- a/sofic/automata/rfsa.py +++ /dev/null @@ -1,35 +0,0 @@ -"""Residual and canonical RFSA automata.""" - -from __future__ import annotations - -from typing import TYPE_CHECKING, Any - -from sofic.automata.languages.base import RegularLanguage -from sofic.automata.nfa import NFA - -if TYPE_CHECKING: - from sofic.automata.observation import ObservationTable - - -class ResidualFiniteStateAutomaton(NFA): - """NFA whose states accept residual languages of the recognized language.""" - - def validate(self) -> None: - super().validate() - # Phase 2: verify each state's right language is in Res(L(R)) - - -class CanonicalRFSA(ResidualFiniteStateAutomaton): - """Canonical residual finite-state automaton R(L).""" - - @classmethod - def from_language(cls, language: RegularLanguage | NFA, **kwargs: Any) -> CanonicalRFSA: - from sofic.automata.canonical_extraction import canonical_rfsa_from_language - - return canonical_rfsa_from_language(language) - - @classmethod - def from_observation_table(cls, table: ObservationTable, **kwargs: Any) -> CanonicalRFSA: - from sofic.automata.canonical_extraction import observation_to_canonical_rfsa - - return observation_to_canonical_rfsa(table) diff --git a/sofic/automata/transducer_operations.py b/sofic/automata/transducer_operations.py index 0bb528c..bc56090 100644 --- a/sofic/automata/transducer_operations.py +++ b/sofic/automata/transducer_operations.py @@ -15,7 +15,7 @@ from sofic.graph import ATTR_EMISSION, ATTR_OUTPUT, ATTR_PROB, ATTR_SYMBOL, EPSILON -def cartesian_product_gg( +def generator_product( generators: Sequence[HiddenMarkovModel], *, create_using: type[MealyHMM] | None = None, @@ -59,7 +59,7 @@ def cartesian_product_gg( return result -def cartesian_product_tt( +def transducer_product( transducers: Sequence[MealyMachine], *, create_using: type[MealyMachine] | None = None, @@ -100,7 +100,7 @@ def cartesian_product_tt( return result -def compose_tt( +def compose_transducers( transducers: Sequence[MealyMachine], *, complete: bool = True, @@ -109,18 +109,20 @@ def compose_tt( ) -> MealyMachine: """Serially compose transducers. - ``compose_tt((t0, t1))`` returns the transducer that feeds ``t0``'s output + ``compose_transducers((t0, t1))`` returns the transducer that feeds ``t0``'s output into ``t1``. State labels are tuples ordered like the input transducers. """ if not transducers: raise ValueError("at least one transducer is required") result = transducers[0].copy() for transducer in transducers[1:]: - result = _compose_pair_tt(result, transducer, complete=complete, create_using=create_using, normalize=normalize) + result = _compose_transducer_pair( + result, transducer, complete=complete, create_using=create_using, normalize=normalize + ) return result -def compose_tg( +def compose_transducer_generator( transducer: MealyMachine, generator: HiddenMarkovModel, *, @@ -191,7 +193,7 @@ def transduce_generator( create_using: type[MealyHMM] | None = None, ) -> MealyHMM: """Return the output-only generator induced by driving ``transducer`` with ``generator``.""" - return compose_tg( + return compose_transducer_generator( transducer, generator, complete=complete, @@ -201,7 +203,7 @@ def transduce_generator( ) -def _compose_pair_tt( +def _compose_transducer_pair( left: MealyMachine, right: MealyMachine, *, @@ -342,9 +344,9 @@ def _prob(data: dict[str, Any]) -> float: __all__ = [ "ERROR_STATE", "ERROR_SYMBOL", - "cartesian_product_gg", - "cartesian_product_tt", - "compose_tg", - "compose_tt", + "generator_product", + "transducer_product", + "compose_transducer_generator", + "compose_transducers", "transduce_generator", ] diff --git a/sofic/automata/transducer_simulation.py b/sofic/automata/transducer_simulation.py index cd700d5..b6fd9c8 100644 --- a/sofic/automata/transducer_simulation.py +++ b/sofic/automata/transducer_simulation.py @@ -50,12 +50,7 @@ def step(config: tuple[Hashable, tuple[Any, ...]], symbol: Any) -> Iterator[tupl for transition in transducer.graph.out_transitions(state): if transition.data.get(ATTR_SYMBOL) != symbol: continue - if moore: - extended = output_prefix + _state_output(transducer, transition.target) - else: - out = transition.data.get(ATTR_OUTPUT) - extended = output_prefix + ((out,) if out is not None and out is not EPSILON else ()) - yield (transition.target, extended) + yield (transition.target, output_prefix + _emitted(transducer, transition, moore=moore)) stack = simulate_configs( initial, @@ -85,12 +80,7 @@ def _epsilon_closure( for transition in transducer.graph.out_transitions(state): if transition.data.get(ATTR_SYMBOL, EPSILON) is not EPSILON: continue - if moore: - extended = output_prefix + _state_output(transducer, transition.target) - else: - out = transition.data.get(ATTR_OUTPUT) - extended = output_prefix + ((out,) if out is not None and out is not EPSILON else ()) - config = (transition.target, extended) + config = (transition.target, output_prefix + _emitted(transducer, transition, moore=moore)) if config not in closure: closure.add(config) stack.append(config) @@ -131,20 +121,14 @@ def _has_productive_epsilon_cycle( if target not in component: continue for edge_data in eps_graph.get_edge_data(source, target).values(): - transition = edge_data["transition"] - if _epsilon_edge_is_productive(transducer, transition.target, transition.data, moore=moore): + if _emitted(transducer, edge_data["transition"], moore=moore): return True return False -def _epsilon_edge_is_productive( - transducer: Transducer, - target: Hashable, - data: dict[str, Any], - *, - moore: bool, -) -> bool: +def _emitted(transducer: Transducer, transition: Any, *, moore: bool) -> tuple[Any, ...]: + """Output appended when taking ``transition`` (Moore: the entered state's output).""" if moore: - return bool(_state_output(transducer, target)) - output = data.get(ATTR_OUTPUT) - return output is not None and output is not EPSILON + return _state_output(transducer, transition.target) + out = transition.data.get(ATTR_OUTPUT) + return (out,) if out is not None and out is not EPSILON else () diff --git a/sofic/automata/transducers.py b/sofic/automata/transducers.py index ffaa28c..817a788 100644 --- a/sofic/automata/transducers.py +++ b/sofic/automata/transducers.py @@ -175,15 +175,15 @@ def output_machine(self, *, build: bool = False) -> Any: def compose(self, other: MealyMachine, **kwargs: Any) -> MealyMachine: """Return the serial composition ``other`` after this transducer.""" - from sofic.automata.transducer_operations import compose_tt + from sofic.automata.transducer_operations import compose_transducers - return compose_tt((self, other), **kwargs) + return compose_transducers((self, other), **kwargs) def joint_machine(self, generator: Any, **kwargs: Any) -> Any: """Return the joint input/output generator induced by ``generator``.""" - from sofic.automata.transducer_operations import compose_tg + from sofic.automata.transducer_operations import compose_transducer_generator - return compose_tg(self, generator, joint=True, **kwargs) + return compose_transducer_generator(self, generator, joint=True, **kwargs) def transduce_generator(self, generator: Any, **kwargs: Any) -> Any: """Return the output generator induced by driving this transducer.""" @@ -191,11 +191,11 @@ def transduce_generator(self, generator: Any, **kwargs: Any) -> Any: return transduce_generator(self, generator, **kwargs) - def to_sofic_relation(self) -> Any: - """Return the topological support as a product-alphabet sofic relation.""" - from sofic.shifts.sofic_relation import SoficRelation + def to_product_alphabet_shift(self) -> Any: + """Return the topological support as a :class:`~sofic.shifts.ProductAlphabetShift`.""" + from sofic.shifts.product_alphabet_shift import ProductAlphabetShift - return SoficRelation.from_transducer(self) + return ProductAlphabetShift.from_transducer(self) def to_textile_system(self) -> Any: """Return this transducer as a textile system (Nasu 1995).""" diff --git a/sofic/automata/vpa.py b/sofic/automata/vpa.py deleted file mode 100644 index 088cdcd..0000000 --- a/sofic/automata/vpa.py +++ /dev/null @@ -1,1373 +0,0 @@ -"""Visibly pushdown automata.""" - -from __future__ import annotations - -from collections import deque -from collections.abc import Hashable, Iterable, Mapping, Sequence -from typing import Any - -from sofic.base import StateMachine -from sofic.exceptions import NonDeterministicError -from sofic.graph import ( - ATTR_KIND, - ATTR_STACK_SYMBOL, - ATTR_SYMBOL, - KIND_CALL, - KIND_INTERNAL, - KIND_RETURN, -) - -_MISSING = object() - - -class VisiblyPushdownAutomaton(StateMachine): - """Standard 1-stack VPA with call / return / internal input partition.""" - - input_alphabet: frozenset[Any] - call_alphabet: frozenset[Any] - return_alphabet: frozenset[Any] - internal_alphabet: frozenset[Any] - stack_alphabet: frozenset[Any] - bottom_stack_symbol: Any | None - initial_state: Hashable | None - accepting_states: frozenset[Hashable] - - def __init__( - self, - input_alphabet: frozenset[Any] | None = None, - call_alphabet: frozenset[Any] | None = None, - return_alphabet: frozenset[Any] | None = None, - internal_alphabet: frozenset[Any] | None = None, - stack_alphabet: frozenset[Any] | None = None, - bottom_stack_symbol: Any | None = None, - initial_state: Hashable | None = None, - accepting_states: frozenset[Hashable] | None = None, - **kwargs: Any, - ) -> None: - super().__init__(**kwargs) - self.call_alphabet = call_alphabet if call_alphabet is not None else frozenset() - self.return_alphabet = return_alphabet if return_alphabet is not None else frozenset() - self.internal_alphabet = internal_alphabet if internal_alphabet is not None else frozenset() - self.stack_alphabet = stack_alphabet if stack_alphabet is not None else frozenset() - self.bottom_stack_symbol = bottom_stack_symbol - self.input_alphabet = ( - input_alphabet - if input_alphabet is not None - else self.call_alphabet | self.return_alphabet | self.internal_alphabet - ) - self.initial_state = initial_state - self.accepting_states = accepting_states if accepting_states is not None else frozenset() - - def validate(self) -> None: - partition = self.call_alphabet | self.return_alphabet | self.internal_alphabet - self._require( - len(self.call_alphabet) + len(self.return_alphabet) + len(self.internal_alphabet) == len(partition), - "call, return, and internal alphabets must be disjoint", - ) - self._require(partition == self.input_alphabet, "input alphabet must equal partition of call/return/internal") - if self.bottom_stack_symbol is not None: - self._require( - self.bottom_stack_symbol in self.stack_alphabet, "bottom_stack_symbol must be in stack alphabet" - ) - if self.initial_state is not None: - self._require(self.graph.has_state(self.initial_state), "missing initial state") - for state in self.accepting_states: - self._require(self.graph.has_state(state), f"missing accepting state {state!r}") - for transition in self.transitions(): - kind = transition.data.get(ATTR_KIND) - symbol = transition.data.get(ATTR_SYMBOL) - self._require(kind in {KIND_CALL, KIND_RETURN, KIND_INTERNAL}, f"invalid VPA kind {kind!r}") - if symbol is not None: - if kind == KIND_CALL: - self._require(symbol in self.call_alphabet, f"{symbol!r} not in call alphabet") - stack_sym = transition.data.get(ATTR_STACK_SYMBOL) - self._require(stack_sym in self.stack_alphabet, "call edge requires stack_symbol in stack alphabet") - self._require( - stack_sym != self.bottom_stack_symbol, - "call edge cannot push the bottom_stack_symbol", - ) - elif kind == KIND_RETURN: - self._require(symbol in self.return_alphabet, f"{symbol!r} not in return alphabet") - stack_sym = transition.data.get(ATTR_STACK_SYMBOL) - if stack_sym is not None: - self._require(stack_sym in self.stack_alphabet, "return stack_symbol must be in stack alphabet") - else: - self._require(symbol in self.internal_alphabet, f"{symbol!r} not in internal alphabet") - - def add_call_transition( - self, - source: Hashable, - target: Hashable, - symbol: Any, - stack_symbol: Any, - **attrs: Any, - ) -> int: - """Add a call transition that pushes ``stack_symbol``.""" - data = {**attrs, ATTR_KIND: KIND_CALL, ATTR_SYMBOL: symbol, ATTR_STACK_SYMBOL: stack_symbol} - return self.graph.add_transition(source, target, **data) - - def add_return_transition( - self, - source: Hashable, - target: Hashable, - symbol: Any, - stack_symbol: Any | None = None, - **attrs: Any, - ) -> int: - """Add a return transition. - - If ``stack_symbol`` is omitted, the transition is a wildcard over - non-bottom stack symbols. This preserves the historical unguarded - return behavior of :class:`VisiblyPushdownAutomaton`. - """ - data = {**attrs, ATTR_KIND: KIND_RETURN, ATTR_SYMBOL: symbol} - if stack_symbol is not None: - data[ATTR_STACK_SYMBOL] = stack_symbol - return self.graph.add_transition(source, target, **data) - - def add_internal_transition(self, source: Hashable, target: Hashable, symbol: Any, **attrs: Any) -> int: - """Add an internal transition.""" - data = {**attrs, ATTR_KIND: KIND_INTERNAL, ATTR_SYMBOL: symbol} - return self.graph.add_transition(source, target, **data) - - def call_transition_map(self) -> dict[tuple[Hashable, Any], tuple[Hashable, Any]]: - """Return deterministic call transitions keyed by ``(state, symbol)``.""" - result: dict[tuple[Hashable, Any], tuple[Hashable, Any]] = {} - for transition in self.transitions(): - if transition.data.get(ATTR_KIND) != KIND_CALL: - continue - symbol = transition.data.get(ATTR_SYMBOL) - if symbol is None: - continue - key = (transition.source, symbol) - value = (transition.target, transition.data.get(ATTR_STACK_SYMBOL)) - if key in result and result[key] != value: - raise NonDeterministicError(f"non-deterministic call transition on {key}") - result[key] = value - return result - - def return_transition_map(self) -> dict[tuple[Hashable, Any, Any | None], Hashable]: - """Return deterministic return transitions keyed by ``(state, symbol, stack_symbol)``.""" - result: dict[tuple[Hashable, Any, Any | None], Hashable] = {} - for transition in self.transitions(): - if transition.data.get(ATTR_KIND) != KIND_RETURN: - continue - symbol = transition.data.get(ATTR_SYMBOL) - if symbol is None: - continue - key = (transition.source, symbol, transition.data.get(ATTR_STACK_SYMBOL)) - value = transition.target - if key in result and result[key] != value: - raise NonDeterministicError(f"non-deterministic return transition on {key}") - result[key] = value - return result - - def internal_transition_map(self) -> dict[tuple[Hashable, Any], Hashable]: - """Return deterministic internal transitions keyed by ``(state, symbol)``.""" - result: dict[tuple[Hashable, Any], Hashable] = {} - for transition in self.transitions(): - if transition.data.get(ATTR_KIND) != KIND_INTERNAL: - continue - symbol = transition.data.get(ATTR_SYMBOL) - if symbol is None: - continue - key = (transition.source, symbol) - value = transition.target - if key in result and result[key] != value: - raise NonDeterministicError(f"non-deterministic internal transition on {key}") - result[key] = value - return result - - def recognizes(self, word: Sequence[Any]) -> bool: - from sofic.automata.vpa_simulation import recognizes_vpa - - return recognizes_vpa(self, word) - - def union(self, other: VisiblyPushdownAutomaton) -> CompositeVisiblyPushdownAutomaton: - """Return a VPA recognizer for the union with ``other``.""" - return union_vpa(self, other) - - def intersection(self, other: VisiblyPushdownAutomaton) -> CompositeVisiblyPushdownAutomaton: - """Return a VPA recognizer for the intersection with ``other``.""" - return intersection_vpa(self, other) - - def intersect(self, other: VisiblyPushdownAutomaton) -> CompositeVisiblyPushdownAutomaton: - """Alias for :meth:`intersection`.""" - return self.intersection(other) - - def complement(self) -> CompositeVisiblyPushdownAutomaton: - """Return a VPA recognizer for the complement over this visible alphabet.""" - return complement_vpa(self) - - def difference(self, other: VisiblyPushdownAutomaton) -> CompositeVisiblyPushdownAutomaton: - """Return a VPA recognizer for this language minus ``other``.""" - return difference_vpa(self, other) - - def concat(self, other: VisiblyPushdownAutomaton) -> CompositeVisiblyPushdownAutomaton: - """Return a VPA recognizer for concatenation with ``other``.""" - return concat_vpa(self, other) - - def concatenate(self, other: VisiblyPushdownAutomaton) -> CompositeVisiblyPushdownAutomaton: - """Alias for :meth:`concat`.""" - return self.concat(other) - - def kleene_star(self) -> CompositeVisiblyPushdownAutomaton: - """Return a VPA recognizer for the Kleene star of this language.""" - return kleene_star_vpa(self) - - def star(self) -> CompositeVisiblyPushdownAutomaton: - """Alias for :meth:`kleene_star`.""" - return self.kleene_star() - - -class CompositeVisiblyPushdownAutomaton(VisiblyPushdownAutomaton): - """Lazy VPA language expression built from standard closure operations. - - Composite VPAs keep exact language semantics for operations whose concrete - graph construction would otherwise need a larger normalization pass. They - still expose the regular VPA membership API through :meth:`recognizes`. - """ - - operation: str - operands: tuple[VisiblyPushdownAutomaton, ...] - - def __init__( - self, - *, - operation: str, - operands: Iterable[VisiblyPushdownAutomaton], - ) -> None: - operands = tuple(operands) - if not operands: - raise ValueError("CompositeVisiblyPushdownAutomaton requires at least one operand") - call_alphabet, return_alphabet, internal_alphabet = _merge_visible_alphabets(operands) - super().__init__( - call_alphabet=call_alphabet, - return_alphabet=return_alphabet, - internal_alphabet=internal_alphabet, - stack_alphabet=frozenset(), - ) - self.operation = operation - self.operands = operands - - def validate(self) -> None: - self._require( - self.operation in {"union", "intersection", "complement", "difference", "concat", "kleene_star"}, - f"unknown composite VPA operation {self.operation!r}", - ) - if self.operation in {"complement", "kleene_star"}: - self._require(len(self.operands) == 1, f"{self.operation} requires one operand") - elif self.operation in {"difference", "concat"}: - self._require(len(self.operands) == 2, f"{self.operation} requires two operands") - else: - self._require(len(self.operands) >= 2, f"{self.operation} requires at least two operands") - for operand in self.operands: - operand.validate() - call_alphabet, return_alphabet, internal_alphabet = _merge_visible_alphabets(self.operands) - self._require(call_alphabet == self.call_alphabet, "composite call alphabet is stale") - self._require(return_alphabet == self.return_alphabet, "composite return alphabet is stale") - self._require(internal_alphabet == self.internal_alphabet, "composite internal alphabet is stale") - - def recognizes(self, word: Sequence[Any]) -> bool: - word = tuple(word) - if any(symbol not in self.input_alphabet for symbol in word): - return False - if self.operation == "union": - return any(operand.recognizes(word) for operand in self.operands) - if self.operation == "intersection": - return all(operand.recognizes(word) for operand in self.operands) - if self.operation == "complement": - return not self.operands[0].recognizes(word) - if self.operation == "difference": - return self.operands[0].recognizes(word) and not self.operands[1].recognizes(word) - if self.operation == "concat": - left, right = self.operands - return any( - left.recognizes(word[:index]) and right.recognizes(word[index:]) for index in range(len(word) + 1) - ) - if self.operation == "kleene_star": - operand = self.operands[0] - accepted = [False] * (len(word) + 1) - accepted[0] = True - for end in range(1, len(word) + 1): - accepted[end] = any(accepted[start] and operand.recognizes(word[start:end]) for start in range(end)) - return accepted[-1] - raise ValueError(f"unknown composite VPA operation {self.operation!r}") - - -def union_vpa( - left: VisiblyPushdownAutomaton, - right: VisiblyPushdownAutomaton, - *rest: VisiblyPushdownAutomaton, -) -> CompositeVisiblyPushdownAutomaton: - """Return a VPA recognizer for the union of the operands.""" - return CompositeVisiblyPushdownAutomaton(operation="union", operands=(left, right, *rest)) - - -def intersection_vpa( - left: VisiblyPushdownAutomaton, - right: VisiblyPushdownAutomaton, - *rest: VisiblyPushdownAutomaton, -) -> CompositeVisiblyPushdownAutomaton: - """Return a VPA recognizer for the intersection of the operands.""" - return CompositeVisiblyPushdownAutomaton(operation="intersection", operands=(left, right, *rest)) - - -def complement_vpa(vpa: VisiblyPushdownAutomaton) -> CompositeVisiblyPushdownAutomaton: - """Return a VPA recognizer for complement over ``vpa``'s visible alphabet.""" - return CompositeVisiblyPushdownAutomaton(operation="complement", operands=(vpa,)) - - -def difference_vpa( - left: VisiblyPushdownAutomaton, - right: VisiblyPushdownAutomaton, -) -> CompositeVisiblyPushdownAutomaton: - """Return a VPA recognizer for ``left`` minus ``right``.""" - return CompositeVisiblyPushdownAutomaton(operation="difference", operands=(left, right)) - - -def concat_vpa( - left: VisiblyPushdownAutomaton, - right: VisiblyPushdownAutomaton, -) -> CompositeVisiblyPushdownAutomaton: - """Return a VPA recognizer for language concatenation.""" - return CompositeVisiblyPushdownAutomaton(operation="concat", operands=(left, right)) - - -def kleene_star_vpa(vpa: VisiblyPushdownAutomaton) -> CompositeVisiblyPushdownAutomaton: - """Return a VPA recognizer for Kleene star.""" - return CompositeVisiblyPushdownAutomaton(operation="kleene_star", operands=(vpa,)) - - -def _merge_visible_alphabets( - vpas: Iterable[VisiblyPushdownAutomaton], -) -> tuple[frozenset[Any], frozenset[Any], frozenset[Any]]: - call_symbols: set[Any] = set() - return_symbols: set[Any] = set() - internal_symbols: set[Any] = set() - owners: dict[Any, str] = {} - for vpa in vpas: - for kind, symbols in ( - ("call", vpa.call_alphabet), - ("return", vpa.return_alphabet), - ("internal", vpa.internal_alphabet), - ): - for symbol in symbols: - existing = owners.get(symbol) - if existing is not None and existing != kind: - raise ValueError(f"symbol {symbol!r} is both {existing} and {kind}") - owners[symbol] = kind - if kind == "call": - call_symbols.add(symbol) - elif kind == "return": - return_symbols.add(symbol) - else: - internal_symbols.add(symbol) - return frozenset(call_symbols), frozenset(return_symbols), frozenset(internal_symbols) - - -class DeterministicVisiblyPushdownAutomaton(VisiblyPushdownAutomaton): - """VPA with at most one enabled transition for each visible configuration.""" - - def validate(self) -> None: - super().validate() - if self.initial_state is None: - raise NonDeterministicError("deterministic VPA requires an initial state") - self._check_determinism() - - def _check_determinism(self) -> None: - call_keys: set[tuple[Hashable, Any]] = set() - internal_keys: set[tuple[Hashable, Any]] = set() - return_keys: set[tuple[Hashable, Any, Any]] = set() - wildcard_returns: set[tuple[Hashable, Any]] = set() - - for transition in self.transitions(): - kind = transition.data.get(ATTR_KIND) - symbol = transition.data.get(ATTR_SYMBOL) - if symbol is None: - continue - if kind == KIND_CALL: - key = (transition.source, symbol) - if key in call_keys: - raise NonDeterministicError(f"non-deterministic call transition on {key}") - call_keys.add(key) - elif kind == KIND_INTERNAL: - key = (transition.source, symbol) - if key in internal_keys: - raise NonDeterministicError(f"non-deterministic internal transition on {key}") - internal_keys.add(key) - elif kind == KIND_RETURN: - stack_symbol = transition.data.get(ATTR_STACK_SYMBOL) - wildcard_key = (transition.source, symbol) - if stack_symbol is None: - if wildcard_key in wildcard_returns: - raise NonDeterministicError(f"duplicate wildcard return transition on {wildcard_key}") - if any(source == transition.source and ret == symbol for source, ret, _stack in return_keys): - raise NonDeterministicError( - f"wildcard return overlaps guarded return on {wildcard_key}", - ) - wildcard_returns.add(wildcard_key) - else: - key = (transition.source, symbol, stack_symbol) - if wildcard_key in wildcard_returns: - raise NonDeterministicError( - f"guarded return overlaps wildcard return on {wildcard_key}", - ) - if key in return_keys: - raise NonDeterministicError(f"duplicate guarded return transition on {key}") - return_keys.add(key) - - def add_call_transition( - self, - source: Hashable, - target: Hashable, - symbol: Any, - stack_symbol: Any, - **attrs: Any, - ) -> int: - for transition in self.graph.out_transitions(source): - if transition.data.get(ATTR_KIND) == KIND_CALL and transition.data.get(ATTR_SYMBOL) == symbol: - raise NonDeterministicError(f"non-deterministic call transition on {(source, symbol)}") - return super().add_call_transition(source, target, symbol, stack_symbol, **attrs) - - def add_return_transition( - self, - source: Hashable, - target: Hashable, - symbol: Any, - stack_symbol: Any | None = None, - **attrs: Any, - ) -> int: - for transition in self.graph.out_transitions(source): - if transition.data.get(ATTR_KIND) != KIND_RETURN or transition.data.get(ATTR_SYMBOL) != symbol: - continue - existing_stack = transition.data.get(ATTR_STACK_SYMBOL) - if existing_stack is None or stack_symbol is None or existing_stack == stack_symbol: - raise NonDeterministicError(f"non-deterministic return transition on {(source, symbol)}") - return super().add_return_transition(source, target, symbol, stack_symbol, **attrs) - - def add_internal_transition(self, source: Hashable, target: Hashable, symbol: Any, **attrs: Any) -> int: - for transition in self.graph.out_transitions(source): - if transition.data.get(ATTR_KIND) == KIND_INTERNAL and transition.data.get(ATTR_SYMBOL) == symbol: - raise NonDeterministicError(f"non-deterministic internal transition on {(source, symbol)}") - return super().add_internal_transition(source, target, symbol, **attrs) - - def call_successor(self, state: Hashable, symbol: Any) -> tuple[Hashable, Any] | None: - """Return ``(target, pushed_stack_symbol)`` for a deterministic call.""" - return self.call_transition_map().get((state, symbol)) - - def internal_successor(self, state: Hashable, symbol: Any) -> Hashable | None: - """Return the deterministic internal successor, if present.""" - return self.internal_transition_map().get((state, symbol)) - - def return_successor(self, state: Hashable, symbol: Any, stack_symbol: Any) -> Hashable | None: - """Return the deterministic return successor for ``stack_symbol``, if present.""" - transitions = self.return_transition_map() - explicit = transitions.get((state, symbol, stack_symbol), _MISSING) - if explicit is not _MISSING: - return explicit - wildcard = transitions.get((state, symbol, None), _MISSING) - if wildcard is not _MISSING: - return wildcard - return None - - @classmethod - def from_vpa(cls, vpa: VisiblyPushdownAutomaton) -> DeterministicVisiblyPushdownAutomaton: - """Copy ``vpa`` into a deterministic VPA and validate determinism.""" - result = cls( - input_alphabet=vpa.input_alphabet, - call_alphabet=vpa.call_alphabet, - return_alphabet=vpa.return_alphabet, - internal_alphabet=vpa.internal_alphabet, - stack_alphabet=vpa.stack_alphabet, - bottom_stack_symbol=vpa.bottom_stack_symbol, - initial_state=vpa.initial_state, - accepting_states=vpa.accepting_states, - graph=vpa.graph.copy(), - ) - result.validate() - return result - - -class CallDrivenAutomaton(DeterministicVisiblyPushdownAutomaton): - """Deterministic modular VPA whose call target depends only on the call symbol.""" - - modules: dict[Hashable, frozenset[Hashable]] - base_module: Hashable - call_partition: dict[Any, Hashable] - call_entries: dict[Any, Hashable] - - def __init__( - self, - *, - modules: Mapping[Hashable, Iterable[Hashable]] | None = None, - base_module: Hashable = 0, - call_partition: Mapping[Any, Hashable] | None = None, - call_entries: Mapping[Any, Hashable] | None = None, - **kwargs: Any, - ) -> None: - super().__init__(**kwargs) - self.modules = _normalize_modules(modules) - self.base_module = base_module - self.call_partition = dict(call_partition or {}) - self.call_entries = dict(call_entries or {}) - - def validate(self) -> None: - super().validate() - state_modules = self._validate_modules() - self._validate_call_partition() - self._validate_internal_transitions_stay_in_module(state_modules) - self._validate_call_driven_transitions() - - def call_entry_map(self) -> dict[Any, Hashable]: - """Return configured or inferred entries for each call symbol with transitions.""" - entries = dict(self.call_entries) - for transition in self.transitions(): - if transition.data.get(ATTR_KIND) != KIND_CALL: - continue - symbol = transition.data.get(ATTR_SYMBOL) - if symbol is None: - continue - existing = entries.get(symbol, _MISSING) - if existing is _MISSING: - entries[symbol] = transition.target - elif existing != transition.target: - raise NonDeterministicError(f"call target for {symbol!r} depends on source state") - return entries - - @classmethod - def minimize( - cls, - vpa: VisiblyPushdownAutomaton, - *, - modules: Mapping[Hashable, Iterable[Hashable]] | None = None, - call_partition: Mapping[Any, Hashable] | None = None, - base_module: Hashable | None = None, - call_entries: Mapping[Any, Hashable] | None = None, - ) -> CallDrivenAutomaton: - """Return the module-aware deterministic quotient as a CDA.""" - return _minimize_modular_vpa( - cls, - vpa, - modules=modules, - call_partition=call_partition, - base_module=base_module, - call_entries=call_entries, - entry_states=None, - form="cda", - ) - - def _validate_modules(self) -> dict[Hashable, Hashable]: - self._require(bool(self.modules), "modular VPA requires modules") - self._require(self.base_module in self.modules, "base_module must be present in modules") - all_states = set(self.states()) - seen: dict[Hashable, Hashable] = {} - for module, states in self.modules.items(): - self._require(bool(states), f"module {module!r} must contain at least one state") - for state in states: - self._require(state in all_states, f"module {module!r} contains unknown state {state!r}") - self._require(state not in seen, f"state {state!r} appears in multiple modules") - seen[state] = module - self._require(set(seen) == all_states, "modules must cover exactly the VPA states") - if self.initial_state is not None: - self._require( - self.initial_state in self.modules[self.base_module], - "initial_state must lie in the base module", - ) - return seen - - def _validate_call_partition(self) -> None: - missing = self.call_alphabet - set(self.call_partition) - extra = set(self.call_partition) - self.call_alphabet - self._require(not missing, f"call_partition missing calls {sorted(missing, key=repr)!r}") - self._require(not extra, f"call_partition contains non-call symbols {sorted(extra, key=repr)!r}") - for symbol, module in self.call_partition.items(): - self._require(module in self.modules, f"call {symbol!r} targets unknown module {module!r}") - - def _validate_internal_transitions_stay_in_module(self, state_modules: Mapping[Hashable, Hashable]) -> None: - for transition in self.transitions(): - if transition.data.get(ATTR_KIND) == KIND_INTERNAL: - self._require( - state_modules[transition.source] == state_modules[transition.target], - "internal transitions must stay inside one module", - ) - - def _validate_call_driven_transitions(self) -> None: - entries = self.call_entry_map() - for symbol, target in entries.items(): - module = self.call_partition[symbol] - self._require(target in self.modules[module], f"call {symbol!r} entry is not in module {module!r}") - for transition in self.transitions(): - if transition.data.get(ATTR_KIND) != KIND_CALL: - continue - symbol = transition.data.get(ATTR_SYMBOL) - if symbol is None: - continue - self._require( - transition.target == entries[symbol], - f"call target for {symbol!r} must be independent of source state", - ) - - -class MultipleEntryVisiblyPushdownAutomaton(CallDrivenAutomaton): - """Modular VPA with multiple module entries and source-determined call pushes.""" - - entry_states: dict[Hashable, frozenset[Hashable]] - - def __init__( - self, - *, - entry_states: Mapping[Hashable, Iterable[Hashable] | Hashable] | None = None, - **kwargs: Any, - ) -> None: - super().__init__(**kwargs) - self.entry_states = _normalize_multi_entries(entry_states) - - def validate(self) -> None: - super().validate() - self._validate_entry_states() - self._validate_call_targets_are_entries() - self._validate_source_determined_pushes() - - @classmethod - def minimize( - cls, - vpa: VisiblyPushdownAutomaton, - *, - modules: Mapping[Hashable, Iterable[Hashable]] | None = None, - call_partition: Mapping[Any, Hashable] | None = None, - base_module: Hashable | None = None, - entry_states: Mapping[Hashable, Iterable[Hashable] | Hashable] | None = None, - call_entries: Mapping[Any, Hashable] | None = None, - ) -> MultipleEntryVisiblyPushdownAutomaton: - """Return the module-aware deterministic quotient as an MEVPA.""" - return _minimize_modular_vpa( - cls, - vpa, - modules=modules, - call_partition=call_partition, - base_module=base_module, - call_entries=call_entries, - entry_states=entry_states, - form="mevpa", - ) - - def _validate_entry_states(self) -> None: - self._require(bool(self.entry_states), "MEVPA requires entry_states") - for module, states in self.entry_states.items(): - self._require(module in self.modules, f"entry_states has unknown module {module!r}") - self._require(bool(states), f"module {module!r} must have at least one entry") - for state in states: - self._require(state in self.modules[module], f"entry {state!r} is not in module {module!r}") - - def _validate_call_targets_are_entries(self) -> None: - entries = self.call_entry_map() - for symbol, target in entries.items(): - module = self.call_partition[symbol] - self._require( - target in self.entry_states.get(module, frozenset()), - f"call {symbol!r} must enter one of module {module!r}'s entries", - ) - - def _validate_source_determined_pushes(self) -> None: - pushed_by_source: dict[Hashable, Any] = {} - for transition in self.transitions(): - if transition.data.get(ATTR_KIND) != KIND_CALL: - continue - pushed = transition.data.get(ATTR_STACK_SYMBOL) - existing = pushed_by_source.get(transition.source, _MISSING) - if existing is _MISSING: - pushed_by_source[transition.source] = pushed - else: - self._require(existing == pushed, "MEVPA call push must depend only on the source state") - - -class SingleEntryVisiblyPushdownAutomaton(CallDrivenAutomaton): - """Modular VPA with one distinguished entry per non-base module.""" - - entry_states: dict[Hashable, Hashable] - - def __init__( - self, - *, - entry_states: Mapping[Hashable, Hashable] | None = None, - **kwargs: Any, - ) -> None: - super().__init__(**kwargs) - self.entry_states = dict(entry_states or {}) - - def validate(self) -> None: - super().validate() - self._validate_single_entries() - self._validate_single_entry_calls() - - @classmethod - def minimize( - cls, - vpa: VisiblyPushdownAutomaton, - *, - call_partition: Mapping[Any, Hashable] | None = None, - modules: Mapping[Hashable, Iterable[Hashable]] | None = None, - base_module: Hashable | None = None, - entry_states: Mapping[Hashable, Hashable] | None = None, - call_entries: Mapping[Any, Hashable] | None = None, - ) -> SingleEntryVisiblyPushdownAutomaton: - """Return the module-aware deterministic quotient as an SEVPA. - - A fixed call partition and module structure are required. General VPA - minimization is intentionally not attempted here. - """ - return _minimize_modular_vpa( - cls, - vpa, - modules=modules, - call_partition=call_partition, - base_module=base_module, - call_entries=call_entries, - entry_states=entry_states, - form="sevpa", - ) - - def _validate_single_entries(self) -> None: - missing = set(self.modules) - {self.base_module} - set(self.entry_states) - self._require(not missing, f"SEVPA missing entries for modules {sorted(missing, key=repr)!r}") - for module, state in self.entry_states.items(): - self._require(module in self.modules, f"entry_states has unknown module {module!r}") - self._require(state in self.modules[module], f"entry {state!r} is not in module {module!r}") - - def _validate_single_entry_calls(self) -> None: - for transition in self.transitions(): - if transition.data.get(ATTR_KIND) != KIND_CALL: - continue - symbol = transition.data.get(ATTR_SYMBOL) - if symbol is None: - continue - module = self.call_partition[symbol] - expected_entry = self.entry_states.get(module) - self._require( - transition.target == expected_entry, - f"SEVPA call {symbol!r} must enter module {module!r}'s single entry", - ) - self._require( - transition.data.get(ATTR_STACK_SYMBOL) == (transition.source, symbol), - "SEVPA call stack symbols must be (caller_state, call_symbol)", - ) - - -class CanonicalVisiblyPushdownAutomaton(DeterministicVisiblyPushdownAutomaton): - """Canonical VPA built from the finite Myhill-Nerode summary algebra.""" - - summary_representatives: dict[Hashable, tuple[int | None, ...]] - - def __init__( - self, - *, - summary_representatives: Mapping[Hashable, tuple[int | None, ...]] | None = None, - **kwargs: Any, - ) -> None: - super().__init__(**kwargs) - self.summary_representatives = dict(summary_representatives or {}) - - @classmethod - def from_vpa(cls, vpa: VisiblyPushdownAutomaton) -> CanonicalVisiblyPushdownAutomaton: - """Build the Myhill-Nerode canonical deterministic VPA for ``vpa``. - - The construction is finite for deterministic VPAs because states are - summary classes of well-matched factors. With an empty call alphabet, - this specializes to the usual minimal DFA right-congruence construction. - """ - det = DeterministicVisiblyPushdownAutomaton.from_vpa(vpa) - algebra = _SummaryAlgebra.from_vpa(det) - return algebra.to_canonical_vpa(cls, det) - - @classmethod - def minimize(cls, vpa: VisiblyPushdownAutomaton) -> CanonicalVisiblyPushdownAutomaton: - """Alias for :meth:`from_vpa`.""" - return cls.from_vpa(vpa) - - -def _normalize_modules(modules: Mapping[Hashable, Iterable[Hashable]] | None) -> dict[Hashable, frozenset[Hashable]]: - if modules is None: - return {} - return {module: frozenset(states) for module, states in modules.items()} - - -def _normalize_multi_entries( - entry_states: Mapping[Hashable, Iterable[Hashable] | Hashable] | None, -) -> dict[Hashable, frozenset[Hashable]]: - if entry_states is None: - return {} - result: dict[Hashable, frozenset[Hashable]] = {} - for module, states in entry_states.items(): - if isinstance(states, frozenset | set | list): - result[module] = frozenset(states) - else: - result[module] = frozenset({states}) - return result - - -def _metadata_or_argument(vpa: VisiblyPushdownAutomaton, name: str, value: Any, default: Any) -> Any: - if value is not None: - return value - return getattr(vpa, name, default) - - -def _deterministic_view(vpa: VisiblyPushdownAutomaton) -> DeterministicVisiblyPushdownAutomaton: - return DeterministicVisiblyPushdownAutomaton.from_vpa(vpa) - - -def _minimize_modular_vpa( - target_cls: type[CallDrivenAutomaton], - vpa: VisiblyPushdownAutomaton, - *, - modules: Mapping[Hashable, Iterable[Hashable]] | None, - call_partition: Mapping[Any, Hashable] | None, - base_module: Hashable | None, - call_entries: Mapping[Any, Hashable] | None, - entry_states: Mapping[Hashable, Any] | None, - form: str, -) -> Any: - modules = _metadata_or_argument(vpa, "modules", modules, None) - call_partition = _metadata_or_argument(vpa, "call_partition", call_partition, None) - base_module = _metadata_or_argument(vpa, "base_module", base_module, 0) - call_entries = _metadata_or_argument(vpa, "call_entries", call_entries, None) - entry_states = _metadata_or_argument(vpa, "entry_states", entry_states, None) - - if modules is None or call_partition is None: - raise NotImplementedError("modular VPA minimization requires fixed modules and call_partition") - - det = _deterministic_view(vpa) - modules = _normalize_modules(modules) - call_partition = dict(call_partition) - call_entries = dict(call_entries or _infer_call_entries(det, call_partition)) - entry_states = _infer_entry_states(form, modules, base_module, call_partition, call_entries, entry_states, det) - - source = _construct_modular_view( - target_cls, - det, - modules=modules, - base_module=base_module, - call_partition=call_partition, - call_entries=call_entries, - entry_states=entry_states, - ) - source.validate() - - partition = _refine_modular_partition(source, modules) - return _quotient_modular_vpa( - target_cls, - source, - partition, - modules=modules, - base_module=base_module, - call_partition=call_partition, - call_entries=call_entries, - entry_states=entry_states, - form=form, - ) - - -def _construct_modular_view( - target_cls: type[CallDrivenAutomaton], - det: DeterministicVisiblyPushdownAutomaton, - *, - modules: Mapping[Hashable, frozenset[Hashable]], - base_module: Hashable, - call_partition: Mapping[Any, Hashable], - call_entries: Mapping[Any, Hashable], - entry_states: Mapping[Hashable, Any], -) -> CallDrivenAutomaton: - kwargs = { - "input_alphabet": det.input_alphabet, - "call_alphabet": det.call_alphabet, - "return_alphabet": det.return_alphabet, - "internal_alphabet": det.internal_alphabet, - "stack_alphabet": det.stack_alphabet, - "bottom_stack_symbol": det.bottom_stack_symbol, - "initial_state": det.initial_state, - "accepting_states": det.accepting_states, - "graph": det.graph.copy(), - "modules": modules, - "base_module": base_module, - "call_partition": call_partition, - "call_entries": call_entries, - } - if issubclass(target_cls, (SingleEntryVisiblyPushdownAutomaton, MultipleEntryVisiblyPushdownAutomaton)): - kwargs["entry_states"] = entry_states - return target_cls(**kwargs) - - -def _infer_call_entries( - det: DeterministicVisiblyPushdownAutomaton, - call_partition: Mapping[Any, Hashable], -) -> dict[Any, Hashable]: - entries: dict[Any, Hashable] = {} - for (_source, symbol), (target, _stack) in det.call_transition_map().items(): - if symbol not in call_partition: - raise NotImplementedError(f"call_partition missing call symbol {symbol!r}") - existing = entries.get(symbol, _MISSING) - if existing is _MISSING: - entries[symbol] = target - elif existing != target: - raise NotImplementedError(f"call target for {symbol!r} depends on source state") - return entries - - -def _infer_entry_states( - form: str, - modules: Mapping[Hashable, frozenset[Hashable]], - base_module: Hashable, - call_partition: Mapping[Any, Hashable], - call_entries: Mapping[Any, Hashable], - entry_states: Mapping[Hashable, Any] | None, - det: DeterministicVisiblyPushdownAutomaton, -) -> dict[Hashable, Any]: - if entry_states is not None: - if form == "mevpa": - return _normalize_multi_entries(entry_states) - return dict(entry_states) - - by_module: dict[Hashable, set[Hashable]] = {module: set() for module in modules} - if det.initial_state is not None and base_module in by_module: - by_module[base_module].add(det.initial_state) - for symbol, entry in call_entries.items(): - by_module[call_partition[symbol]].add(entry) - - if form == "mevpa": - return {module: frozenset(states) for module, states in by_module.items() if states} - if form == "sevpa": - result: dict[Hashable, Hashable] = {} - for module, states in by_module.items(): - if module == base_module: - continue - if len(states) != 1: - raise NotImplementedError("SEVPA minimization requires one inferred entry per non-base module") - result[module] = next(iter(states)) - return result - return {} - - -def _refine_modular_partition( - vpa: CallDrivenAutomaton, - modules: Mapping[Hashable, frozenset[Hashable]], -) -> list[frozenset[Hashable]]: - partition: list[frozenset[Hashable]] = [] - for _module, states in sorted(modules.items(), key=lambda item: repr(item[0])): - accepting = frozenset(states & vpa.accepting_states) - rejecting = frozenset(states - vpa.accepting_states) - if accepting: - partition.append(accepting) - if rejecting: - partition.append(rejecting) - - changed = True - while changed: - changed = False - block_of = _block_map(partition) - context_groups = _stack_context_groups(vpa, block_of) - new_partition: list[frozenset[Hashable]] = [] - for block in partition: - pieces: dict[tuple[Any, ...], set[Hashable]] = {} - for state in block: - signature = _modular_state_signature(vpa, state, block_of, context_groups) - pieces.setdefault(signature, set()).add(state) - if len(pieces) > 1: - changed = True - new_partition.extend(frozenset(piece) for piece in pieces.values()) - partition = new_partition - return partition - - -def _block_map(partition: Sequence[frozenset[Hashable]]) -> dict[Hashable, frozenset[Hashable]]: - return {state: block for block in partition for state in block} - - -def _stack_context_groups( - vpa: DeterministicVisiblyPushdownAutomaton, - block_of: Mapping[Hashable, Hashable], -) -> list[tuple[Any, tuple[Any, ...]]]: - stack_symbols = {stack for _key, (_target, stack) in vpa.call_transition_map().items()} - if vpa.bottom_stack_symbol is not None: - stack_symbols.add(vpa.bottom_stack_symbol) - grouped: dict[Any, set[Any]] = {} - for stack_symbol in stack_symbols: - canonical = _canonical_stack_symbol(stack_symbol, block_of) - grouped.setdefault(canonical, set()).add(stack_symbol) - return [(canonical, tuple(sorted(actuals, key=repr))) for canonical, actuals in sorted(grouped.items(), key=repr)] - - -def _modular_state_signature( - vpa: CallDrivenAutomaton, - state: Hashable, - block_of: Mapping[Hashable, Hashable], - context_groups: Sequence[tuple[Any, tuple[Any, ...]]], -) -> tuple[Any, ...]: - state_modules = {state: module for module, states in vpa.modules.items() for state in states} - call_map = vpa.call_transition_map() - internal_map = vpa.internal_transition_map() - return_map = vpa.return_transition_map() - - internal = tuple( - ( - symbol, - None if (target := internal_map.get((state, symbol))) is None else block_of[target], - ) - for symbol in sorted(vpa.internal_alphabet, key=repr) - ) - calls = tuple( - ( - symbol, - None - if (call := call_map.get((state, symbol))) is None - else (block_of[call[0]], _canonical_stack_symbol(call[1], block_of)), - ) - for symbol in sorted(vpa.call_alphabet, key=repr) - ) - returns = [] - for symbol in sorted(vpa.return_alphabet, key=repr): - for canonical_stack, actual_stacks in context_groups: - targets = [] - for stack_symbol in actual_stacks: - target = return_map.get((state, symbol, stack_symbol), return_map.get((state, symbol, None))) - targets.append(None if target is None else block_of[target]) - returns.append((symbol, canonical_stack, tuple(sorted(set(targets), key=repr)))) - - return ( - state_modules[state], - state in vpa.accepting_states, - internal, - calls, - tuple(returns), - ) - - -def _quotient_modular_vpa( - target_cls: type[CallDrivenAutomaton], - source: CallDrivenAutomaton, - partition: Sequence[frozenset[Hashable]], - *, - modules: Mapping[Hashable, frozenset[Hashable]], - base_module: Hashable, - call_partition: Mapping[Any, Hashable], - call_entries: Mapping[Any, Hashable], - entry_states: Mapping[Hashable, Any], - form: str, -) -> Any: - block_of = _block_map(partition) - blocks = list(partition) - stack_alphabet: set[Any] = set() - if source.bottom_stack_symbol is not None: - stack_alphabet.add(source.bottom_stack_symbol) - transitions: set[tuple[Hashable, Hashable, Any, Any, Any | None]] = set() - stack_symbol_rewrite: dict[Any, set[Any]] = {} - - for transition in source.transitions(): - if transition.data.get(ATTR_KIND) != KIND_CALL: - continue - symbol = transition.data.get(ATTR_SYMBOL) - stack_symbol = transition.data.get(ATTR_STACK_SYMBOL) - quotient_stack_symbol = _quotient_call_stack_symbol( - form, - block_of[transition.source], - symbol, - stack_symbol, - block_of, - ) - stack_symbol_rewrite.setdefault(stack_symbol, set()).add(quotient_stack_symbol) - - for block in blocks: - representative = min(block, key=repr) - source_block = block_of[representative] - for transition in source.graph.out_transitions(representative): - kind = transition.data.get(ATTR_KIND) - symbol = transition.data.get(ATTR_SYMBOL) - target_block = block_of[transition.target] - stack_symbol = transition.data.get(ATTR_STACK_SYMBOL) - if kind == KIND_CALL: - stack_symbol = _quotient_call_stack_symbol(form, source_block, symbol, stack_symbol, block_of) - stack_alphabet.add(stack_symbol) - elif kind == KIND_RETURN and stack_symbol is not None: - rewritten = stack_symbol_rewrite.get(stack_symbol, {_canonical_stack_symbol(stack_symbol, block_of)}) - for rewritten_stack_symbol in rewritten: - stack_alphabet.add(rewritten_stack_symbol) - transitions.add((source_block, target_block, kind, symbol, rewritten_stack_symbol)) - continue - transitions.add((source_block, target_block, kind, symbol, stack_symbol)) - - quotient_modules = { - module: frozenset(block for block in blocks if block & states) - for module, states in modules.items() - if any(block & states for block in blocks) - } - quotient_call_entries = {symbol: block_of[state] for symbol, state in call_entries.items() if state in block_of} - quotient_entry_states = _quotient_entry_states(form, entry_states, block_of) - - kwargs = { - "input_alphabet": source.input_alphabet, - "call_alphabet": source.call_alphabet, - "return_alphabet": source.return_alphabet, - "internal_alphabet": source.internal_alphabet, - "stack_alphabet": frozenset(stack_alphabet), - "bottom_stack_symbol": source.bottom_stack_symbol, - "initial_state": block_of[source.initial_state], - "accepting_states": frozenset(block for block in blocks if block & source.accepting_states), - "modules": quotient_modules, - "base_module": base_module, - "call_partition": dict(call_partition), - "call_entries": quotient_call_entries, - } - if issubclass(target_cls, (SingleEntryVisiblyPushdownAutomaton, MultipleEntryVisiblyPushdownAutomaton)): - kwargs["entry_states"] = quotient_entry_states - - result = target_cls(**kwargs) - for block in blocks: - result.graph.add_state(block) - for source_block, target_block, kind, symbol, stack_symbol in sorted(transitions, key=repr): - if kind == KIND_CALL: - result.add_call_transition(source_block, target_block, symbol, stack_symbol) - elif kind == KIND_RETURN: - result.add_return_transition(source_block, target_block, symbol, stack_symbol) - elif kind == KIND_INTERNAL: - result.add_internal_transition(source_block, target_block, symbol) - result.validate() - return result - - -def _quotient_call_stack_symbol( - form: str, - source_block: Hashable, - symbol: Any, - stack_symbol: Any, - block_of: Mapping[Hashable, Hashable], -) -> Any: - if form == "sevpa": - return (source_block, symbol) - if form == "mevpa": - return source_block - return _canonical_stack_symbol(stack_symbol, block_of) - - -def _quotient_entry_states( - form: str, - entry_states: Mapping[Hashable, Any], - block_of: Mapping[Hashable, Hashable], -) -> dict[Hashable, Any]: - if form == "mevpa": - normalized = _normalize_multi_entries(entry_states) - return { - module: frozenset(block_of[state] for state in states if state in block_of) - for module, states in normalized.items() - } - if form == "sevpa": - return {module: block_of[state] for module, state in entry_states.items() if state in block_of} - return {} - - -def _canonical_stack_symbol(stack_symbol: Any, block_of: Mapping[Hashable, Hashable]) -> Any: - if stack_symbol in block_of: - return block_of[stack_symbol] - if isinstance(stack_symbol, tuple): - return tuple(_canonical_stack_symbol(part, block_of) for part in stack_symbol) - return stack_symbol - - -class _SummaryAlgebra: - def __init__( - self, - *, - state_order: tuple[Hashable, ...], - summaries: frozenset[tuple[int | None, ...]], - identity: tuple[int | None, ...], - internal_summaries: Mapping[Any, tuple[int | None, ...]], - class_of: Mapping[tuple[int | None, ...], int], - representatives: Mapping[int, tuple[int | None, ...]], - ) -> None: - self.state_order = state_order - self.summaries = summaries - self.identity = identity - self.internal_summaries = dict(internal_summaries) - self.class_of = dict(class_of) - self.representatives = dict(representatives) - - @classmethod - def from_vpa(cls, vpa: DeterministicVisiblyPushdownAutomaton) -> _SummaryAlgebra: - state_order = tuple(sorted(vpa.states(), key=repr)) - state_index = {state: index for index, state in enumerate(state_order)} - internal_summaries = { - symbol: _internal_summary(vpa, state_order, state_index, symbol) - for symbol in sorted(vpa.internal_alphabet, key=repr) - } - identity = tuple(range(len(state_order))) - summaries = _close_summary_algebra(vpa, state_order, state_index, identity, internal_summaries) - class_of, representatives = _quotient_summaries(vpa, state_order, state_index, summaries, identity) - return cls( - state_order=state_order, - summaries=frozenset(summaries), - identity=identity, - internal_summaries=internal_summaries, - class_of=class_of, - representatives=representatives, - ) - - def to_canonical_vpa( - self, - cls: type[CanonicalVisiblyPushdownAutomaton], - source: DeterministicVisiblyPushdownAutomaton, - ) -> CanonicalVisiblyPushdownAutomaton: - identity_class = self.class_of[self.identity] - states = frozenset(self.representatives) - stack_alphabet = frozenset( - (summary_class, symbol) for summary_class in states for symbol in source.call_alphabet - ) - accepting_states = frozenset( - summary_class - for summary_class, summary in self.representatives.items() - if _summary_accepts(source, self.state_order, summary) - ) - result = cls( - input_alphabet=source.input_alphabet, - call_alphabet=source.call_alphabet, - return_alphabet=source.return_alphabet, - internal_alphabet=source.internal_alphabet, - stack_alphabet=stack_alphabet, - bottom_stack_symbol=None, - initial_state=identity_class, - accepting_states=accepting_states, - summary_representatives=self.representatives, - ) - for state in states: - result.graph.add_state(state) - - for summary_class, summary in sorted(self.representatives.items(), key=repr): - for symbol, internal in sorted(self.internal_summaries.items(), key=lambda item: repr(item[0])): - target_summary = _compose_summary(summary, internal) - result.add_internal_transition(summary_class, self.class_of[target_summary], symbol) - for symbol in sorted(source.call_alphabet, key=repr): - result.add_call_transition(summary_class, identity_class, symbol, (summary_class, symbol)) - - for inner_class, inner in sorted(self.representatives.items(), key=repr): - for outer_class, outer in sorted(self.representatives.items(), key=repr): - for call_symbol in sorted(source.call_alphabet, key=repr): - for return_symbol in sorted(source.return_alphabet, key=repr): - wrapped = _wrap_summary(source, self.state_order, inner, call_symbol, return_symbol) - target = _compose_summary(outer, wrapped) - result.add_return_transition( - inner_class, - self.class_of[target], - return_symbol, - (outer_class, call_symbol), - ) - - result.validate() - return result - - -def _internal_summary( - vpa: DeterministicVisiblyPushdownAutomaton, - state_order: Sequence[Hashable], - state_index: Mapping[Hashable, int], - symbol: Any, -) -> tuple[int | None, ...]: - transitions = vpa.internal_transition_map() - summary: list[int | None] = [] - for state in state_order: - target = transitions.get((state, symbol)) - summary.append(None if target is None else state_index[target]) - return tuple(summary) - - -def _close_summary_algebra( - vpa: DeterministicVisiblyPushdownAutomaton, - state_order: Sequence[Hashable], - state_index: Mapping[Hashable, int], - identity: tuple[int | None, ...], - internal_summaries: Mapping[Any, tuple[int | None, ...]], -) -> set[tuple[int | None, ...]]: - summaries = {identity, *internal_summaries.values()} - queue: deque[tuple[int | None, ...]] = deque(sorted(summaries, key=repr)) - while queue: - summary = queue.popleft() - current = list(summaries) - candidates: list[tuple[int | None, ...]] = [] - for other in current: - candidates.append(_compose_summary(summary, other)) - candidates.append(_compose_summary(other, summary)) - for call_symbol in sorted(vpa.call_alphabet, key=repr): - for return_symbol in sorted(vpa.return_alphabet, key=repr): - candidates.append(_wrap_summary(vpa, state_order, summary, call_symbol, return_symbol)) - for candidate in candidates: - if candidate not in summaries: - summaries.add(candidate) - queue.append(candidate) - return summaries - - -def _compose_summary( - first: tuple[int | None, ...], - second: tuple[int | None, ...], -) -> tuple[int | None, ...]: - return tuple(None if state is None else second[state] for state in first) - - -def _wrap_summary( - vpa: DeterministicVisiblyPushdownAutomaton, - state_order: Sequence[Hashable], - inner: tuple[int | None, ...], - call_symbol: Any, - return_symbol: Any, -) -> tuple[int | None, ...]: - state_index = {state: index for index, state in enumerate(state_order)} - call_map = vpa.call_transition_map() - result: list[int | None] = [] - for state in state_order: - call = call_map.get((state, call_symbol)) - if call is None: - result.append(None) - continue - call_target, stack_symbol = call - inner_target_index = inner[state_index[call_target]] - if inner_target_index is None: - result.append(None) - continue - return_target = vpa.return_successor(state_order[inner_target_index], return_symbol, stack_symbol) - result.append(None if return_target is None else state_index[return_target]) - return tuple(result) - - -def _quotient_summaries( - vpa: DeterministicVisiblyPushdownAutomaton, - state_order: Sequence[Hashable], - state_index: Mapping[Hashable, int], - summaries: set[tuple[int | None, ...]], - identity: tuple[int | None, ...], -) -> tuple[dict[tuple[int | None, ...], int], dict[int, tuple[int | None, ...]]]: - contexts = tuple(sorted(summaries, key=repr)) - signatures = { - summary: tuple(_summary_accepts(vpa, state_order, _compose_summary(summary, context)) for context in contexts) - for summary in summaries - } - identity_signature = signatures[identity] - ordered_signatures = sorted( - set(signatures.values()), key=lambda signature: (signature != identity_signature, signature) - ) - signature_class = {signature: index for index, signature in enumerate(ordered_signatures)} - class_of = {summary: signature_class[signature] for summary, signature in signatures.items()} - representatives = { - index: min( - (summary for summary, signature in signatures.items() if signature_class[signature] == index), key=repr - ) - for index in signature_class.values() - } - return class_of, representatives - - -def _summary_accepts( - vpa: DeterministicVisiblyPushdownAutomaton, - state_order: Sequence[Hashable], - summary: tuple[int | None, ...], -) -> bool: - if vpa.initial_state is None: - return False - initial_index = {state: index for index, state in enumerate(state_order)}[vpa.initial_state] - target = summary[initial_index] - return target is not None and state_order[target] in vpa.accepting_states diff --git a/sofic/automata/vpa/__init__.py b/sofic/automata/vpa/__init__.py new file mode 100644 index 0000000..c58d1b7 --- /dev/null +++ b/sofic/automata/vpa/__init__.py @@ -0,0 +1,26 @@ +"""Visibly pushdown automata, their constructions, and simulation.""" + +from sofic.automata.vpa.base import VisiblyPushdownAutomaton +from sofic.automata.vpa.canonical import CanonicalVisiblyPushdownAutomaton +from sofic.automata.vpa.deterministic import DeterministicVisiblyPushdownAutomaton +from sofic.automata.vpa.modular import ( + ModularVisiblyPushdownAutomaton, + MultipleEntryVisiblyPushdownAutomaton, + SingleEntryVisiblyPushdownAutomaton, +) +from sofic.automata.vpa.operations import BOTTOM, NormalVPA, to_multiple_entry, to_single_entry +from sofic.automata.vpa.simulation import recognizes_vpa + +__all__ = [ + "BOTTOM", + "ModularVisiblyPushdownAutomaton", + "CanonicalVisiblyPushdownAutomaton", + "DeterministicVisiblyPushdownAutomaton", + "MultipleEntryVisiblyPushdownAutomaton", + "NormalVPA", + "SingleEntryVisiblyPushdownAutomaton", + "VisiblyPushdownAutomaton", + "recognizes_vpa", + "to_multiple_entry", + "to_single_entry", +] diff --git a/sofic/automata/vpa/base.py b/sofic/automata/vpa/base.py new file mode 100644 index 0000000..ca20f65 --- /dev/null +++ b/sofic/automata/vpa/base.py @@ -0,0 +1,251 @@ +"""Visibly pushdown automata.""" + +from __future__ import annotations + +from collections.abc import Hashable, Sequence +from typing import TYPE_CHECKING, Any + +from sofic.automata.vpa import operations as vc +from sofic.base import StateMachine +from sofic.exceptions import NonDeterministicError +from sofic.graph import ( + ATTR_KIND, + ATTR_STACK_SYMBOL, + ATTR_SYMBOL, + KIND_CALL, + KIND_INTERNAL, + KIND_RETURN, +) + +if TYPE_CHECKING: + from sofic.automata.vpa.deterministic import DeterministicVisiblyPushdownAutomaton + +_MISSING = object() + + +class VisiblyPushdownAutomaton(StateMachine): + """Standard 1-stack VPA with call / return / internal input partition.""" + + input_alphabet: frozenset[Any] + call_alphabet: frozenset[Any] + return_alphabet: frozenset[Any] + internal_alphabet: frozenset[Any] + stack_alphabet: frozenset[Any] + bottom_stack_symbol: Any | None + initial_state: Hashable | None + accepting_states: frozenset[Hashable] + + def __init__( + self, + input_alphabet: frozenset[Any] | None = None, + call_alphabet: frozenset[Any] | None = None, + return_alphabet: frozenset[Any] | None = None, + internal_alphabet: frozenset[Any] | None = None, + stack_alphabet: frozenset[Any] | None = None, + bottom_stack_symbol: Any | None = None, + initial_state: Hashable | None = None, + accepting_states: frozenset[Hashable] | None = None, + **kwargs: Any, + ) -> None: + super().__init__(**kwargs) + self.call_alphabet = call_alphabet if call_alphabet is not None else frozenset() + self.return_alphabet = return_alphabet if return_alphabet is not None else frozenset() + self.internal_alphabet = internal_alphabet if internal_alphabet is not None else frozenset() + self.stack_alphabet = stack_alphabet if stack_alphabet is not None else frozenset() + self.bottom_stack_symbol = bottom_stack_symbol + self.input_alphabet = ( + input_alphabet + if input_alphabet is not None + else self.call_alphabet | self.return_alphabet | self.internal_alphabet + ) + self.initial_state = initial_state + self.accepting_states = accepting_states if accepting_states is not None else frozenset() + + def validate(self) -> None: + partition = self.call_alphabet | self.return_alphabet | self.internal_alphabet + self._require( + len(self.call_alphabet) + len(self.return_alphabet) + len(self.internal_alphabet) == len(partition), + "call, return, and internal alphabets must be disjoint", + ) + self._require(partition == self.input_alphabet, "input alphabet must equal partition of call/return/internal") + if self.bottom_stack_symbol is not None: + self._require( + self.bottom_stack_symbol in self.stack_alphabet, "bottom_stack_symbol must be in stack alphabet" + ) + if self.initial_state is not None: + self._require(self.graph.has_state(self.initial_state), "missing initial state") + for state in self.accepting_states: + self._require(self.graph.has_state(state), f"missing accepting state {state!r}") + for transition in self.transitions(): + kind = transition.data.get(ATTR_KIND) + symbol = transition.data.get(ATTR_SYMBOL) + self._require(kind in {KIND_CALL, KIND_RETURN, KIND_INTERNAL}, f"invalid VPA kind {kind!r}") + if symbol is not None: + if kind == KIND_CALL: + self._require(symbol in self.call_alphabet, f"{symbol!r} not in call alphabet") + stack_sym = transition.data.get(ATTR_STACK_SYMBOL) + self._require(stack_sym in self.stack_alphabet, "call edge requires stack_symbol in stack alphabet") + self._require( + stack_sym != self.bottom_stack_symbol, + "call edge cannot push the bottom_stack_symbol", + ) + elif kind == KIND_RETURN: + self._require(symbol in self.return_alphabet, f"{symbol!r} not in return alphabet") + stack_sym = transition.data.get(ATTR_STACK_SYMBOL) + if stack_sym is not None: + self._require(stack_sym in self.stack_alphabet, "return stack_symbol must be in stack alphabet") + else: + self._require(symbol in self.internal_alphabet, f"{symbol!r} not in internal alphabet") + + def add_call_transition( + self, + source: Hashable, + target: Hashable, + symbol: Any, + stack_symbol: Any, + **attrs: Any, + ) -> int: + """Add a call transition that pushes ``stack_symbol``.""" + data = {**attrs, ATTR_KIND: KIND_CALL, ATTR_SYMBOL: symbol, ATTR_STACK_SYMBOL: stack_symbol} + return self.graph.add_transition(source, target, **data) + + def add_return_transition( + self, + source: Hashable, + target: Hashable, + symbol: Any, + stack_symbol: Any | None = None, + **attrs: Any, + ) -> int: + """Add a return transition. + + If ``stack_symbol`` is omitted, the transition is a wildcard: it fires + on every stack symbol, and also on the empty stack (leaving it empty) + when the VPA has a ``bottom_stack_symbol``. A return guarded by the + bottom symbol fires only on the empty stack. + """ + data = {**attrs, ATTR_KIND: KIND_RETURN, ATTR_SYMBOL: symbol} + if stack_symbol is not None: + data[ATTR_STACK_SYMBOL] = stack_symbol + return self.graph.add_transition(source, target, **data) + + def add_internal_transition(self, source: Hashable, target: Hashable, symbol: Any, **attrs: Any) -> int: + """Add an internal transition.""" + data = {**attrs, ATTR_KIND: KIND_INTERNAL, ATTR_SYMBOL: symbol} + return self.graph.add_transition(source, target, **data) + + def call_transition_map(self) -> dict[tuple[Hashable, Any], tuple[Hashable, Any]]: + """Return deterministic call transitions keyed by ``(state, symbol)``.""" + result: dict[tuple[Hashable, Any], tuple[Hashable, Any]] = {} + for transition in self.transitions(): + if transition.data.get(ATTR_KIND) != KIND_CALL: + continue + symbol = transition.data.get(ATTR_SYMBOL) + if symbol is None: + continue + key = (transition.source, symbol) + value = (transition.target, transition.data.get(ATTR_STACK_SYMBOL)) + if key in result and result[key] != value: + raise NonDeterministicError(f"non-deterministic call transition on {key}") + result[key] = value + return result + + def return_transition_map(self) -> dict[tuple[Hashable, Any, Any | None], Hashable]: + """Return deterministic return transitions keyed by ``(state, symbol, stack_symbol)``.""" + result: dict[tuple[Hashable, Any, Any | None], Hashable] = {} + for transition in self.transitions(): + if transition.data.get(ATTR_KIND) != KIND_RETURN: + continue + symbol = transition.data.get(ATTR_SYMBOL) + if symbol is None: + continue + key = (transition.source, symbol, transition.data.get(ATTR_STACK_SYMBOL)) + value = transition.target + if key in result and result[key] != value: + raise NonDeterministicError(f"non-deterministic return transition on {key}") + result[key] = value + return result + + def internal_transition_map(self) -> dict[tuple[Hashable, Any], Hashable]: + """Return deterministic internal transitions keyed by ``(state, symbol)``.""" + result: dict[tuple[Hashable, Any], Hashable] = {} + for transition in self.transitions(): + if transition.data.get(ATTR_KIND) != KIND_INTERNAL: + continue + symbol = transition.data.get(ATTR_SYMBOL) + if symbol is None: + continue + key = (transition.source, symbol) + value = transition.target + if key in result and result[key] != value: + raise NonDeterministicError(f"non-deterministic internal transition on {key}") + result[key] = value + return result + + def recognizes(self, word: Sequence[Any]) -> bool: + from sofic.automata.vpa.simulation import recognizes_vpa + + return recognizes_vpa(self, word) + + def union(self, other: VisiblyPushdownAutomaton) -> VisiblyPushdownAutomaton: + """Return a VPA recognizing the union with ``other``.""" + return _binary(vc.union, self, other) + + def intersection(self, other: VisiblyPushdownAutomaton) -> VisiblyPushdownAutomaton: + """Return a VPA recognizing the intersection with ``other`` (synchronized product).""" + return _binary(vc.intersection, self, other) + + def complement(self) -> DeterministicVisiblyPushdownAutomaton: + """Return a deterministic VPA recognizing the complement over this visible alphabet.""" + from sofic.automata.vpa.deterministic import DeterministicVisiblyPushdownAutomaton + + return vc.denormalize(vc.complement(vc.normalize(self)), DeterministicVisiblyPushdownAutomaton) + + def difference(self, other: VisiblyPushdownAutomaton) -> VisiblyPushdownAutomaton: + """Return a VPA recognizing this language minus ``other``.""" + return _binary(vc.difference, self, other) + + def concat(self, other: VisiblyPushdownAutomaton) -> VisiblyPushdownAutomaton: + """Return a VPA recognizing concatenation with ``other``. + + Each factor is read from an empty stack of its own: a return in the right + factor that would pop a pending call of the left factor is a pending + return of the right factor. + """ + return _binary(vc.concat, self, other) + + def kleene_star(self) -> VisiblyPushdownAutomaton: + """Return a VPA recognizing the Kleene star; each factor starts from its own empty stack.""" + return vc.denormalize(vc.kleene_star(vc.normalize(self))) + + def determinize(self) -> DeterministicVisiblyPushdownAutomaton: + """Return an equivalent complete deterministic VPA :cite:`AlurMadhusudan2009`.""" + return vc.determinize_vpa(self) + + def is_empty(self) -> bool: + """Return whether the language is empty.""" + return vc.is_empty(vc.normalize(self)) + + def accepted_word(self) -> tuple[Any, ...] | None: + """Return a short accepted word, or ``None`` when the language is empty.""" + return vc.accepted_word(vc.normalize(self)) + + def is_universal(self) -> bool: + """Return whether every word over the visible alphabet is accepted.""" + return vc.is_empty(vc.complement(vc.normalize(self))) + + def includes(self, other: VisiblyPushdownAutomaton) -> bool: + """Return whether ``other``'s language is contained in this one.""" + return vc.is_empty(vc.difference(vc.normalize(other), vc.normalize(self))) + + def equivalent(self, other: VisiblyPushdownAutomaton) -> bool: + """Return whether both VPAs recognize the same language.""" + return self.includes(other) and other.includes(self) + + def has_unmatched_word(self) -> bool: + """Return whether some accepted word has a pending call or a pending return.""" + return vc.has_unmatched_word(vc.normalize(self)) + + +def _binary(operation, left: VisiblyPushdownAutomaton, right: VisiblyPushdownAutomaton) -> VisiblyPushdownAutomaton: + return vc.denormalize(operation(vc.normalize(left), vc.normalize(right))) diff --git a/sofic/automata/vpa/canonical.py b/sofic/automata/vpa/canonical.py new file mode 100644 index 0000000..d24e33f --- /dev/null +++ b/sofic/automata/vpa/canonical.py @@ -0,0 +1,280 @@ +"""Canonical visibly pushdown automata from the well-matched summary algebra.""" + +from __future__ import annotations + +from collections import deque +from collections.abc import Hashable, Mapping, Sequence +from typing import Any + +from sofic.automata.vpa.base import VisiblyPushdownAutomaton +from sofic.automata.vpa.deterministic import DeterministicVisiblyPushdownAutomaton +from sofic.exceptions import NonWellMatchedLanguageError + + +class CanonicalVisiblyPushdownAutomaton(DeterministicVisiblyPushdownAutomaton): + """Canonical VPA built from the finite Myhill-Nerode summary algebra.""" + + summary_representatives: dict[Hashable, tuple[int | None, ...]] + + def __init__( + self, + *, + summary_representatives: Mapping[Hashable, tuple[int | None, ...]] | None = None, + **kwargs: Any, + ) -> None: + super().__init__(**kwargs) + self.summary_representatives = dict(summary_representatives or {}) + + @classmethod + def from_vpa(cls, vpa: VisiblyPushdownAutomaton) -> CanonicalVisiblyPushdownAutomaton: + """Build the Myhill-Nerode canonical deterministic VPA of a well-matched language. + + States are classes of the summary algebra of well-matched factors, so + the form is canonical for well-matched languages. Raises + :class:`~sofic.exceptions.NonWellMatchedLanguageError` when ``vpa`` + accepts a word with a pending call or return; general VPLs have no + unique minimal deterministic VPA :cite:`AlurKumarMadhusudanViswanathan2005`. + With an empty call alphabet this is the minimal DFA. + """ + if vpa.has_unmatched_word(): + raise NonWellMatchedLanguageError( + "the canonical VPA is defined for well-matched languages; this one accepts a pending call or return" + ) + det = DeterministicVisiblyPushdownAutomaton.from_vpa(vpa) + return _SummaryAlgebra(det).to_canonical_vpa(cls) + + @classmethod + def minimize(cls, vpa: VisiblyPushdownAutomaton) -> CanonicalVisiblyPushdownAutomaton: + """Alias for :meth:`from_vpa`.""" + return cls.from_vpa(vpa) + + +class _SummaryAlgebra: + """Finite algebra of well-matched summaries with top-level and nested congruences. + + A summary maps each source state to the state reached along a well-matched + word (``None`` when the run dies). Top-level summaries are identified when + every well-matched continuation accepts both or neither; nested summaries + (inside a pending call) are identified when, for every enclosing context, + returning from them lands in the same class. Both partitions are refined + together until internal steps, calls, and returns are well defined on + classes, which makes the quotient canonical. + """ + + def __init__(self, vpa: DeterministicVisiblyPushdownAutomaton) -> None: + self.vpa = vpa + self.state_order = tuple(sorted(vpa.states(), key=repr)) + state_index = {state: index for index, state in enumerate(self.state_order)} + self.internal_summaries = { + symbol: _internal_summary(vpa, self.state_order, state_index, symbol) + for symbol in sorted(vpa.internal_alphabet, key=repr) + } + self.identity = tuple(range(len(self.state_order))) + self.summaries = sorted( + _close_summary_algebra(vpa, self.state_order, state_index, self.identity, self.internal_summaries), + key=repr, + ) + self.calls = sorted(vpa.call_alphabet, key=repr) + self.returns = sorted(vpa.return_alphabet, key=repr) + self._wrap_cache: dict[tuple, tuple] = {} + self.top, self.nested = self._refine() + + def wrap(self, inner: tuple, call: Any, ret: Any) -> tuple: + key = (inner, call, ret) + if key not in self._wrap_cache: + self._wrap_cache[key] = _wrap_summary(self.vpa, self.state_order, inner, call, ret) + return self._wrap_cache[key] + + def accepts(self, summary: tuple) -> bool: + return _summary_accepts(self.vpa, self.state_order, summary) + + def _refine(self) -> tuple[dict[tuple, int], dict[tuple, int]]: + top = {s: int(self.accepts(s)) for s in self.summaries} + nested = dict.fromkeys(self.summaries, 0) + internals = list(self.internal_summaries.values()) + pairs = [(c, r) for c in self.calls for r in self.returns] + while True: + top_signature = { + s: ( + top[s], + tuple(top[_compose_summary(s, a)] for a in internals), + tuple(top[_compose_summary(s, self.wrap(x, c, r))] for x in self.summaries for c, r in pairs), + ) + for s in self.summaries + } + nested_signature = { + s: ( + nested[s], + tuple(nested[_compose_summary(s, a)] for a in internals), + tuple(nested[_compose_summary(s, self.wrap(x, c, r))] for x in self.summaries for c, r in pairs), + tuple( + (top[_compose_summary(o, self.wrap(s, c, r))], nested[_compose_summary(o, self.wrap(s, c, r))]) + for o in self.summaries + for c, r in pairs + ), + ) + for s in self.summaries + } + new_top = _number_blocks(top_signature, first=top_signature[self.identity]) + new_nested = _number_blocks(nested_signature, first=nested_signature[self.identity]) + stable = len(set(new_top.values())) == len(set(top.values())) and len(set(new_nested.values())) == len( + set(nested.values()) + ) + top, nested = new_top, new_nested + if stable: + return top, nested + + def to_canonical_vpa(self, cls: type[CanonicalVisiblyPushdownAutomaton]) -> CanonicalVisiblyPushdownAutomaton: + representative: dict[Hashable, tuple] = {} + for s in self.summaries: + representative.setdefault(self.top[s], s) + representative.setdefault(("nested", self.nested[s]), s) + + def label(summary: tuple, nested: bool) -> Hashable: + return ("nested", self.nested[summary]) if nested else self.top[summary] + + initial = label(self.identity, False) + entry = label(self.identity, True) + reachable: set[Hashable] = set() + internals, calls, returns = set(), set(), set() + pushed: set[tuple[Hashable, Any]] = set() + frontier = [initial] + while frontier: + while frontier: + state = frontier.pop() + if state in reachable: + continue + reachable.add(state) + summary = representative[state] + for symbol, step in self.internal_summaries.items(): + target = label(_compose_summary(summary, step), isinstance(state, tuple)) + internals.add((state, symbol, target)) + frontier.append(target) + for call in self.calls: + calls.add((state, call, entry, (state, call))) + pushed.add((state, call)) + frontier.append(entry) + # Returns pop a pushed (outer, call) from a nested state; both sets + # grow together, so repeat until no new state appears. + for state in [s for s in reachable if isinstance(s, tuple)]: + for outer, call in list(pushed): + for ret in self.returns: + combined = _compose_summary(representative[outer], self.wrap(representative[state], call, ret)) + target = label(combined, isinstance(outer, tuple)) + returns.add((state, ret, (outer, call), target)) + if target not in reachable: + frontier.append(target) + + result = cls( + input_alphabet=self.vpa.input_alphabet, + call_alphabet=self.vpa.call_alphabet, + return_alphabet=self.vpa.return_alphabet, + internal_alphabet=self.vpa.internal_alphabet, + stack_alphabet=frozenset(pushed), + bottom_stack_symbol=None, + initial_state=initial, + accepting_states=frozenset( + s for s in reachable if not isinstance(s, tuple) and self.accepts(representative[s]) + ), + summary_representatives={s: representative[s] for s in reachable}, + ) + for state in sorted(reachable, key=repr): + result.graph.add_state(state) + for source, symbol, target in sorted(internals, key=repr): + result.add_internal_transition(source, target, symbol) + for source, symbol, target, push in sorted(calls, key=repr): + result.add_call_transition(source, target, symbol, push) + for source, symbol, guard, target in sorted(returns, key=repr): + result.add_return_transition(source, target, symbol, guard) + result.validate() + return result + + +def _number_blocks(signatures: Mapping[tuple, Any], *, first: Any) -> dict[tuple, int]: + ordered = sorted(set(signatures.values()), key=lambda sig: (sig != first, repr(sig))) + index = {signature: position for position, signature in enumerate(ordered)} + return {summary: index[signature] for summary, signature in signatures.items()} + + +def _internal_summary( + vpa: DeterministicVisiblyPushdownAutomaton, + state_order: Sequence[Hashable], + state_index: Mapping[Hashable, int], + symbol: Any, +) -> tuple[int | None, ...]: + transitions = vpa.internal_transition_map() + summary: list[int | None] = [] + for state in state_order: + target = transitions.get((state, symbol)) + summary.append(None if target is None else state_index[target]) + return tuple(summary) + + +def _close_summary_algebra( + vpa: DeterministicVisiblyPushdownAutomaton, + state_order: Sequence[Hashable], + state_index: Mapping[Hashable, int], + identity: tuple[int | None, ...], + internal_summaries: Mapping[Any, tuple[int | None, ...]], +) -> set[tuple[int | None, ...]]: + summaries = {identity, *internal_summaries.values()} + queue: deque[tuple[int | None, ...]] = deque(sorted(summaries, key=repr)) + while queue: + summary = queue.popleft() + current = list(summaries) + candidates: list[tuple[int | None, ...]] = [] + for other in current: + candidates.append(_compose_summary(summary, other)) + candidates.append(_compose_summary(other, summary)) + for call_symbol in sorted(vpa.call_alphabet, key=repr): + for return_symbol in sorted(vpa.return_alphabet, key=repr): + candidates.append(_wrap_summary(vpa, state_order, summary, call_symbol, return_symbol)) + for candidate in candidates: + if candidate not in summaries: + summaries.add(candidate) + queue.append(candidate) + return summaries + + +def _compose_summary( + first: tuple[int | None, ...], + second: tuple[int | None, ...], +) -> tuple[int | None, ...]: + return tuple(None if state is None else second[state] for state in first) + + +def _wrap_summary( + vpa: DeterministicVisiblyPushdownAutomaton, + state_order: Sequence[Hashable], + inner: tuple[int | None, ...], + call_symbol: Any, + return_symbol: Any, +) -> tuple[int | None, ...]: + state_index = {state: index for index, state in enumerate(state_order)} + call_map = vpa.call_transition_map() + result: list[int | None] = [] + for state in state_order: + call = call_map.get((state, call_symbol)) + if call is None: + result.append(None) + continue + call_target, stack_symbol = call + inner_target_index = inner[state_index[call_target]] + if inner_target_index is None: + result.append(None) + continue + return_target = vpa.return_successor(state_order[inner_target_index], return_symbol, stack_symbol) + result.append(None if return_target is None else state_index[return_target]) + return tuple(result) + + +def _summary_accepts( + vpa: DeterministicVisiblyPushdownAutomaton, + state_order: Sequence[Hashable], + summary: tuple[int | None, ...], +) -> bool: + if vpa.initial_state is None: + return False + initial_index = {state: index for index, state in enumerate(state_order)}[vpa.initial_state] + target = summary[initial_index] + return target is not None and state_order[target] in vpa.accepting_states diff --git a/sofic/automata/vpa/deterministic.py b/sofic/automata/vpa/deterministic.py new file mode 100644 index 0000000..c1916f9 --- /dev/null +++ b/sofic/automata/vpa/deterministic.py @@ -0,0 +1,166 @@ +"""Deterministic visibly pushdown automata.""" + +from __future__ import annotations + +from collections.abc import Hashable +from typing import Any + +from sofic.automata.vpa import operations as vc +from sofic.automata.vpa.base import _MISSING, VisiblyPushdownAutomaton +from sofic.exceptions import NonDeterministicError +from sofic.graph import ( + ATTR_KIND, + ATTR_STACK_SYMBOL, + ATTR_SYMBOL, + KIND_CALL, + KIND_INTERNAL, + KIND_RETURN, +) + + +class DeterministicVisiblyPushdownAutomaton(VisiblyPushdownAutomaton): + """VPA with at most one enabled transition for each visible configuration.""" + + def validate(self) -> None: + super().validate() + if self.initial_state is None: + raise NonDeterministicError("deterministic VPA requires an initial state") + self._check_determinism() + + def _check_determinism(self) -> None: + call_keys: set[tuple[Hashable, Any]] = set() + internal_keys: set[tuple[Hashable, Any]] = set() + return_keys: set[tuple[Hashable, Any, Any]] = set() + wildcard_returns: set[tuple[Hashable, Any]] = set() + + for transition in self.transitions(): + kind = transition.data.get(ATTR_KIND) + symbol = transition.data.get(ATTR_SYMBOL) + if symbol is None: + continue + if kind == KIND_CALL: + key = (transition.source, symbol) + if key in call_keys: + raise NonDeterministicError(f"non-deterministic call transition on {key}") + call_keys.add(key) + elif kind == KIND_INTERNAL: + key = (transition.source, symbol) + if key in internal_keys: + raise NonDeterministicError(f"non-deterministic internal transition on {key}") + internal_keys.add(key) + elif kind == KIND_RETURN: + stack_symbol = transition.data.get(ATTR_STACK_SYMBOL) + wildcard_key = (transition.source, symbol) + if stack_symbol is None: + if wildcard_key in wildcard_returns: + raise NonDeterministicError(f"duplicate wildcard return transition on {wildcard_key}") + if any(source == transition.source and ret == symbol for source, ret, _stack in return_keys): + raise NonDeterministicError( + f"wildcard return overlaps guarded return on {wildcard_key}", + ) + wildcard_returns.add(wildcard_key) + else: + key = (transition.source, symbol, stack_symbol) + if wildcard_key in wildcard_returns: + raise NonDeterministicError( + f"guarded return overlaps wildcard return on {wildcard_key}", + ) + if key in return_keys: + raise NonDeterministicError(f"duplicate guarded return transition on {key}") + return_keys.add(key) + + def add_call_transition( + self, + source: Hashable, + target: Hashable, + symbol: Any, + stack_symbol: Any, + **attrs: Any, + ) -> int: + for transition in self.graph.out_transitions(source): + if transition.data.get(ATTR_KIND) == KIND_CALL and transition.data.get(ATTR_SYMBOL) == symbol: + raise NonDeterministicError(f"non-deterministic call transition on {(source, symbol)}") + return super().add_call_transition(source, target, symbol, stack_symbol, **attrs) + + def add_return_transition( + self, + source: Hashable, + target: Hashable, + symbol: Any, + stack_symbol: Any | None = None, + **attrs: Any, + ) -> int: + for transition in self.graph.out_transitions(source): + if transition.data.get(ATTR_KIND) != KIND_RETURN or transition.data.get(ATTR_SYMBOL) != symbol: + continue + existing_stack = transition.data.get(ATTR_STACK_SYMBOL) + if existing_stack is None or stack_symbol is None or existing_stack == stack_symbol: + raise NonDeterministicError(f"non-deterministic return transition on {(source, symbol)}") + return super().add_return_transition(source, target, symbol, stack_symbol, **attrs) + + def add_internal_transition(self, source: Hashable, target: Hashable, symbol: Any, **attrs: Any) -> int: + for transition in self.graph.out_transitions(source): + if transition.data.get(ATTR_KIND) == KIND_INTERNAL and transition.data.get(ATTR_SYMBOL) == symbol: + raise NonDeterministicError(f"non-deterministic internal transition on {(source, symbol)}") + return super().add_internal_transition(source, target, symbol, **attrs) + + def call_successor(self, state: Hashable, symbol: Any) -> tuple[Hashable, Any] | None: + """Return ``(target, pushed_stack_symbol)`` for a deterministic call.""" + return self.call_transition_map().get((state, symbol)) + + def internal_successor(self, state: Hashable, symbol: Any) -> Hashable | None: + """Return the deterministic internal successor, if present.""" + return self.internal_transition_map().get((state, symbol)) + + def return_successor(self, state: Hashable, symbol: Any, stack_symbol: Any) -> Hashable | None: + """Return the deterministic return successor for ``stack_symbol``, if present.""" + transitions = self.return_transition_map() + explicit = transitions.get((state, symbol, stack_symbol), _MISSING) + if explicit is not _MISSING: + return explicit + wildcard = transitions.get((state, symbol, None), _MISSING) + if wildcard is not _MISSING: + return wildcard + return None + + @classmethod + def from_vpa(cls, vpa: VisiblyPushdownAutomaton) -> DeterministicVisiblyPushdownAutomaton: + """Return ``vpa`` as a deterministic VPA, determinizing it when needed. + + An already deterministic ``vpa`` is copied with its states unchanged; + otherwise the summary construction of :cite:`AlurMadhusudan2009` is + applied (see :func:`~sofic.automata.vpa.operations.determinize`). + """ + if vpa.initial_state is None or not _is_deterministic(vpa): + return vc.determinize_vpa(vpa) + result = cls( + input_alphabet=vpa.input_alphabet, + call_alphabet=vpa.call_alphabet, + return_alphabet=vpa.return_alphabet, + internal_alphabet=vpa.internal_alphabet, + stack_alphabet=vpa.stack_alphabet, + bottom_stack_symbol=vpa.bottom_stack_symbol, + initial_state=vpa.initial_state, + accepting_states=vpa.accepting_states, + graph=vpa.graph.copy(), + ) + result.validate() + return result + + +def _is_deterministic(vpa: VisiblyPushdownAutomaton) -> bool: + probe = DeterministicVisiblyPushdownAutomaton( + call_alphabet=vpa.call_alphabet, + return_alphabet=vpa.return_alphabet, + internal_alphabet=vpa.internal_alphabet, + stack_alphabet=vpa.stack_alphabet, + bottom_stack_symbol=vpa.bottom_stack_symbol, + initial_state=vpa.initial_state, + accepting_states=vpa.accepting_states, + graph=vpa.graph, + ) + try: + probe._check_determinism() + except NonDeterministicError: + return False + return True diff --git a/sofic/automata/vpa/modular.py b/sofic/automata/vpa/modular.py new file mode 100644 index 0000000..2b41d21 --- /dev/null +++ b/sofic/automata/vpa/modular.py @@ -0,0 +1,686 @@ +"""Modular (call-driven, single- and multiple-entry) visibly pushdown automata.""" + +from __future__ import annotations + +from collections.abc import Hashable, Iterable, Mapping, Sequence +from typing import Any + +from sofic.automata.vpa import operations as vc +from sofic.automata.vpa.base import _MISSING, VisiblyPushdownAutomaton +from sofic.automata.vpa.deterministic import DeterministicVisiblyPushdownAutomaton +from sofic.exceptions import NonDeterministicError +from sofic.graph import ( + ATTR_KIND, + ATTR_STACK_SYMBOL, + ATTR_SYMBOL, + KIND_CALL, + KIND_INTERNAL, + KIND_RETURN, +) + + +class ModularVisiblyPushdownAutomaton(DeterministicVisiblyPushdownAutomaton): + """Deterministic modular VPA whose call target depends only on the call symbol.""" + + modules: dict[Hashable, frozenset[Hashable]] + base_module: Hashable + call_partition: dict[Any, Hashable] + call_entries: dict[Any, Hashable] + + def __init__( + self, + *, + modules: Mapping[Hashable, Iterable[Hashable]] | None = None, + base_module: Hashable = 0, + call_partition: Mapping[Any, Hashable] | None = None, + call_entries: Mapping[Any, Hashable] | None = None, + **kwargs: Any, + ) -> None: + super().__init__(**kwargs) + self.modules = _normalize_modules(modules) + self.base_module = base_module + self.call_partition = dict(call_partition or {}) + self.call_entries = dict(call_entries or {}) + + def validate(self) -> None: + super().validate() + state_modules = self._validate_modules() + self._validate_call_partition() + self._validate_internal_transitions_stay_in_module(state_modules) + self._validate_call_driven_transitions() + + def call_entry_map(self) -> dict[Any, Hashable]: + """Return configured or inferred entries for each call symbol with transitions.""" + entries = dict(self.call_entries) + for transition in self.transitions(): + if transition.data.get(ATTR_KIND) != KIND_CALL: + continue + symbol = transition.data.get(ATTR_SYMBOL) + if symbol is None: + continue + existing = entries.get(symbol, _MISSING) + if existing is _MISSING: + entries[symbol] = transition.target + elif existing != transition.target: + raise NonDeterministicError(f"call target for {symbol!r} depends on source state") + return entries + + @classmethod + def minimize( + cls, + vpa: VisiblyPushdownAutomaton, + *, + modules: Mapping[Hashable, Iterable[Hashable]] | None = None, + call_partition: Mapping[Any, Hashable] | None = None, + base_module: Hashable | None = None, + call_entries: Mapping[Any, Hashable] | None = None, + ) -> ModularVisiblyPushdownAutomaton: + """Return the module-aware deterministic quotient as a CDA.""" + return _minimize_modular_vpa( + cls, + vpa, + modules=modules, + call_partition=call_partition, + base_module=base_module, + call_entries=call_entries, + entry_states=None, + form="cda", + ) + + def _validate_modules(self) -> dict[Hashable, Hashable]: + self._require(bool(self.modules), "modular VPA requires modules") + self._require(self.base_module in self.modules, "base_module must be present in modules") + all_states = set(self.states()) + seen: dict[Hashable, Hashable] = {} + for module, states in self.modules.items(): + self._require(bool(states), f"module {module!r} must contain at least one state") + for state in states: + self._require(state in all_states, f"module {module!r} contains unknown state {state!r}") + self._require(state not in seen, f"state {state!r} appears in multiple modules") + seen[state] = module + self._require(set(seen) == all_states, "modules must cover exactly the VPA states") + if self.initial_state is not None: + self._require( + self.initial_state in self.modules[self.base_module], + "initial_state must lie in the base module", + ) + return seen + + def _validate_call_partition(self) -> None: + missing = self.call_alphabet - set(self.call_partition) + extra = set(self.call_partition) - self.call_alphabet + self._require(not missing, f"call_partition missing calls {sorted(missing, key=repr)!r}") + self._require(not extra, f"call_partition contains non-call symbols {sorted(extra, key=repr)!r}") + for symbol, module in self.call_partition.items(): + self._require(module in self.modules, f"call {symbol!r} targets unknown module {module!r}") + + def _validate_internal_transitions_stay_in_module(self, state_modules: Mapping[Hashable, Hashable]) -> None: + for transition in self.transitions(): + if transition.data.get(ATTR_KIND) == KIND_INTERNAL: + self._require( + state_modules[transition.source] == state_modules[transition.target], + "internal transitions must stay inside one module", + ) + + def _validate_call_driven_transitions(self) -> None: + entries = self.call_entry_map() + for symbol, target in entries.items(): + module = self.call_partition[symbol] + self._require(target in self.modules[module], f"call {symbol!r} entry is not in module {module!r}") + for transition in self.transitions(): + if transition.data.get(ATTR_KIND) != KIND_CALL: + continue + symbol = transition.data.get(ATTR_SYMBOL) + if symbol is None: + continue + self._require( + transition.target == entries[symbol], + f"call target for {symbol!r} must be independent of source state", + ) + + +class MultipleEntryVisiblyPushdownAutomaton(ModularVisiblyPushdownAutomaton): + """Modular VPA with multiple module entries and source-determined call pushes.""" + + entry_states: dict[Hashable, frozenset[Hashable]] + + def __init__( + self, + *, + entry_states: Mapping[Hashable, Iterable[Hashable] | Hashable] | None = None, + **kwargs: Any, + ) -> None: + super().__init__(**kwargs) + self.entry_states = _normalize_multi_entries(entry_states) + + def validate(self) -> None: + super().validate() + self._validate_entry_states() + self._validate_call_targets_are_entries() + self._validate_source_determined_pushes() + + @classmethod + def minimize( + cls, + vpa: VisiblyPushdownAutomaton, + *, + modules: Mapping[Hashable, Iterable[Hashable]] | None = None, + call_partition: Mapping[Any, Hashable] | None = None, + base_module: Hashable | None = None, + entry_states: Mapping[Hashable, Iterable[Hashable] | Hashable] | None = None, + call_entries: Mapping[Any, Hashable] | None = None, + ) -> MultipleEntryVisiblyPushdownAutomaton: + """Return the module-aware deterministic quotient as an MEVPA.""" + return _minimize_modular_vpa( + cls, + vpa, + modules=modules, + call_partition=call_partition, + base_module=base_module, + call_entries=call_entries, + entry_states=entry_states, + form="mevpa", + ) + + def _validate_entry_states(self) -> None: + self._require(bool(self.entry_states), "MEVPA requires entry_states") + for module, states in self.entry_states.items(): + self._require(module in self.modules, f"entry_states has unknown module {module!r}") + self._require(bool(states), f"module {module!r} must have at least one entry") + for state in states: + self._require(state in self.modules[module], f"entry {state!r} is not in module {module!r}") + + def _validate_call_targets_are_entries(self) -> None: + entries = self.call_entry_map() + for symbol, target in entries.items(): + module = self.call_partition[symbol] + self._require( + target in self.entry_states.get(module, frozenset()), + f"call {symbol!r} must enter one of module {module!r}'s entries", + ) + + def _validate_source_determined_pushes(self) -> None: + pushed_by_source: dict[Hashable, Any] = {} + for transition in self.transitions(): + if transition.data.get(ATTR_KIND) != KIND_CALL: + continue + pushed = transition.data.get(ATTR_STACK_SYMBOL) + existing = pushed_by_source.get(transition.source, _MISSING) + if existing is _MISSING: + pushed_by_source[transition.source] = pushed + else: + self._require(existing == pushed, "MEVPA call push must depend only on the source state") + + +class SingleEntryVisiblyPushdownAutomaton(ModularVisiblyPushdownAutomaton): + """Modular VPA with one distinguished entry per non-base module.""" + + entry_states: dict[Hashable, Hashable] + + def __init__( + self, + *, + entry_states: Mapping[Hashable, Hashable] | None = None, + **kwargs: Any, + ) -> None: + super().__init__(**kwargs) + self.entry_states = dict(entry_states or {}) + + def validate(self) -> None: + super().validate() + self._validate_single_entries() + self._validate_single_entry_calls() + + @classmethod + def minimize( + cls, + vpa: VisiblyPushdownAutomaton, + *, + call_partition: Mapping[Any, Hashable] | None = None, + modules: Mapping[Hashable, Iterable[Hashable]] | None = None, + base_module: Hashable | None = None, + entry_states: Mapping[Hashable, Hashable] | None = None, + call_entries: Mapping[Any, Hashable] | None = None, + ) -> SingleEntryVisiblyPushdownAutomaton: + """Return the module-aware deterministic quotient as an SEVPA. + + A fixed call partition and module structure are required. General VPA + minimization is intentionally not attempted here. + """ + return _minimize_modular_vpa( + cls, + vpa, + modules=modules, + call_partition=call_partition, + base_module=base_module, + call_entries=call_entries, + entry_states=entry_states, + form="sevpa", + ) + + def _validate_single_entries(self) -> None: + missing = set(self.modules) - {self.base_module} - set(self.entry_states) + self._require(not missing, f"SEVPA missing entries for modules {sorted(missing, key=repr)!r}") + for module, state in self.entry_states.items(): + self._require(module in self.modules, f"entry_states has unknown module {module!r}") + self._require(state in self.modules[module], f"entry {state!r} is not in module {module!r}") + + def _validate_single_entry_calls(self) -> None: + for transition in self.transitions(): + if transition.data.get(ATTR_KIND) != KIND_CALL: + continue + symbol = transition.data.get(ATTR_SYMBOL) + if symbol is None: + continue + module = self.call_partition[symbol] + expected_entry = self.entry_states.get(module) + self._require( + transition.target == expected_entry, + f"SEVPA call {symbol!r} must enter module {module!r}'s single entry", + ) + self._require( + transition.data.get(ATTR_STACK_SYMBOL) == (transition.source, symbol), + "SEVPA call stack symbols must be (caller_state, call_symbol)", + ) + + +def _normalize_modules(modules: Mapping[Hashable, Iterable[Hashable]] | None) -> dict[Hashable, frozenset[Hashable]]: + if modules is None: + return {} + return {module: frozenset(states) for module, states in modules.items()} + + +def _normalize_multi_entries( + entry_states: Mapping[Hashable, Iterable[Hashable] | Hashable] | None, +) -> dict[Hashable, frozenset[Hashable]]: + if entry_states is None: + return {} + result: dict[Hashable, frozenset[Hashable]] = {} + for module, states in entry_states.items(): + if isinstance(states, frozenset | set | list): + result[module] = frozenset(states) + else: + result[module] = frozenset({states}) + return result + + +def _metadata_or_argument(vpa: VisiblyPushdownAutomaton, name: str, value: Any, default: Any) -> Any: + if value is not None: + return value + return getattr(vpa, name, default) + + +def _deterministic_view(vpa: VisiblyPushdownAutomaton) -> DeterministicVisiblyPushdownAutomaton: + return DeterministicVisiblyPushdownAutomaton.from_vpa(vpa) + + +def _minimize_modular_vpa( + target_cls: type[ModularVisiblyPushdownAutomaton], + vpa: VisiblyPushdownAutomaton, + *, + modules: Mapping[Hashable, Iterable[Hashable]] | None, + call_partition: Mapping[Any, Hashable] | None, + base_module: Hashable | None, + call_entries: Mapping[Any, Hashable] | None, + entry_states: Mapping[Hashable, Any] | None, + form: str, +) -> Any: + modules = _metadata_or_argument(vpa, "modules", modules, None) + call_partition = _metadata_or_argument(vpa, "call_partition", call_partition, None) + base_module = _metadata_or_argument(vpa, "base_module", base_module, 0) + call_entries = _metadata_or_argument(vpa, "call_entries", call_entries, None) + entry_states = _metadata_or_argument(vpa, "entry_states", entry_states, None) + + if modules is None: + convert = vc.to_multiple_entry if form == "mevpa" else vc.to_single_entry + converted = convert(vpa, call_partition) + return _minimize_modular_vpa( + target_cls, + converted, + modules=converted.modules, + call_partition=converted.call_partition, + base_module=converted.base_module, + call_entries=converted.call_entries, + entry_states=getattr(converted, "entry_states", None) if form != "cda" else None, + form=form, + ) + if call_partition is None: + raise ValueError("a call_partition is required when modules are given") + + det = _deterministic_view(vpa) + modules = _normalize_modules(modules) + call_partition = dict(call_partition) + call_entries = dict(call_entries or _infer_call_entries(det, call_partition)) + entry_states = _infer_entry_states(form, modules, base_module, call_partition, call_entries, entry_states, det) + + source = _construct_modular_view( + target_cls, + det, + modules=modules, + base_module=base_module, + call_partition=call_partition, + call_entries=call_entries, + entry_states=entry_states, + ) + source.validate() + + partition = _refine_modular_partition(source, modules) + return _quotient_modular_vpa( + target_cls, + source, + partition, + modules=modules, + base_module=base_module, + call_partition=call_partition, + call_entries=call_entries, + entry_states=entry_states, + form=form, + ) + + +def _construct_modular_view( + target_cls: type[ModularVisiblyPushdownAutomaton], + det: DeterministicVisiblyPushdownAutomaton, + *, + modules: Mapping[Hashable, frozenset[Hashable]], + base_module: Hashable, + call_partition: Mapping[Any, Hashable], + call_entries: Mapping[Any, Hashable], + entry_states: Mapping[Hashable, Any], +) -> ModularVisiblyPushdownAutomaton: + kwargs = { + "input_alphabet": det.input_alphabet, + "call_alphabet": det.call_alphabet, + "return_alphabet": det.return_alphabet, + "internal_alphabet": det.internal_alphabet, + "stack_alphabet": det.stack_alphabet, + "bottom_stack_symbol": det.bottom_stack_symbol, + "initial_state": det.initial_state, + "accepting_states": det.accepting_states, + "graph": det.graph.copy(), + "modules": modules, + "base_module": base_module, + "call_partition": call_partition, + "call_entries": call_entries, + } + if issubclass(target_cls, (SingleEntryVisiblyPushdownAutomaton, MultipleEntryVisiblyPushdownAutomaton)): + kwargs["entry_states"] = entry_states + return target_cls(**kwargs) + + +def _infer_call_entries( + det: DeterministicVisiblyPushdownAutomaton, + call_partition: Mapping[Any, Hashable], +) -> dict[Any, Hashable]: + entries: dict[Any, Hashable] = {} + for (_source, symbol), (target, _stack) in det.call_transition_map().items(): + if symbol not in call_partition: + raise ValueError(f"call_partition missing call symbol {symbol!r}") + existing = entries.get(symbol, _MISSING) + if existing is _MISSING: + entries[symbol] = target + elif existing != target: + raise ValueError( + f"call target for {symbol!r} depends on the source state; omit modules to convert the VPA first" + ) + return entries + + +def _infer_entry_states( + form: str, + modules: Mapping[Hashable, frozenset[Hashable]], + base_module: Hashable, + call_partition: Mapping[Any, Hashable], + call_entries: Mapping[Any, Hashable], + entry_states: Mapping[Hashable, Any] | None, + det: DeterministicVisiblyPushdownAutomaton, +) -> dict[Hashable, Any]: + if entry_states is not None: + if form == "mevpa": + return _normalize_multi_entries(entry_states) + return dict(entry_states) + + by_module: dict[Hashable, set[Hashable]] = {module: set() for module in modules} + if det.initial_state is not None and base_module in by_module: + by_module[base_module].add(det.initial_state) + for symbol, entry in call_entries.items(): + by_module[call_partition[symbol]].add(entry) + + if form == "mevpa": + return {module: frozenset(states) for module, states in by_module.items() if states} + if form == "sevpa": + result: dict[Hashable, Hashable] = {} + for module, states in by_module.items(): + if module == base_module: + continue + if len(states) != 1: + raise ValueError( + "SEVPA minimization requires one entry per non-base module; omit modules to convert the VPA first" + ) + result[module] = next(iter(states)) + return result + return {} + + +def _refine_modular_partition( + vpa: ModularVisiblyPushdownAutomaton, + modules: Mapping[Hashable, frozenset[Hashable]], +) -> list[frozenset[Hashable]]: + partition: list[frozenset[Hashable]] = [] + for _module, states in sorted(modules.items(), key=lambda item: repr(item[0])): + accepting = frozenset(states & vpa.accepting_states) + rejecting = frozenset(states - vpa.accepting_states) + if accepting: + partition.append(accepting) + if rejecting: + partition.append(rejecting) + + changed = True + while changed: + changed = False + block_of = _block_map(partition) + context_groups = _stack_context_groups(vpa, block_of) + new_partition: list[frozenset[Hashable]] = [] + for block in partition: + pieces: dict[tuple[Any, ...], set[Hashable]] = {} + for state in block: + signature = _modular_state_signature(vpa, state, block_of, context_groups) + pieces.setdefault(signature, set()).add(state) + if len(pieces) > 1: + changed = True + new_partition.extend(frozenset(piece) for piece in pieces.values()) + partition = new_partition + return partition + + +def _block_map(partition: Sequence[frozenset[Hashable]]) -> dict[Hashable, frozenset[Hashable]]: + return {state: block for block in partition for state in block} + + +def _stack_context_groups( + vpa: DeterministicVisiblyPushdownAutomaton, + block_of: Mapping[Hashable, Hashable], +) -> list[tuple[Any, tuple[Any, ...]]]: + stack_symbols = {stack for _key, (_target, stack) in vpa.call_transition_map().items()} + if vpa.bottom_stack_symbol is not None: + stack_symbols.add(vpa.bottom_stack_symbol) + grouped: dict[Any, set[Any]] = {} + for stack_symbol in stack_symbols: + canonical = _canonical_stack_symbol(stack_symbol, block_of) + grouped.setdefault(canonical, set()).add(stack_symbol) + return [(canonical, tuple(sorted(actuals, key=repr))) for canonical, actuals in sorted(grouped.items(), key=repr)] + + +def _modular_state_signature( + vpa: ModularVisiblyPushdownAutomaton, + state: Hashable, + block_of: Mapping[Hashable, Hashable], + context_groups: Sequence[tuple[Any, tuple[Any, ...]]], +) -> tuple[Any, ...]: + state_modules = {state: module for module, states in vpa.modules.items() for state in states} + call_map = vpa.call_transition_map() + internal_map = vpa.internal_transition_map() + return_map = vpa.return_transition_map() + + internal = tuple( + ( + symbol, + None if (target := internal_map.get((state, symbol))) is None else block_of[target], + ) + for symbol in sorted(vpa.internal_alphabet, key=repr) + ) + calls = tuple( + ( + symbol, + None + if (call := call_map.get((state, symbol))) is None + else (block_of[call[0]], _canonical_stack_symbol(call[1], block_of)), + ) + for symbol in sorted(vpa.call_alphabet, key=repr) + ) + returns = [] + for symbol in sorted(vpa.return_alphabet, key=repr): + for canonical_stack, actual_stacks in context_groups: + targets = [] + for stack_symbol in actual_stacks: + target = return_map.get((state, symbol, stack_symbol), return_map.get((state, symbol, None))) + targets.append(None if target is None else block_of[target]) + returns.append((symbol, canonical_stack, tuple(sorted(set(targets), key=repr)))) + + return ( + state_modules[state], + state in vpa.accepting_states, + internal, + calls, + tuple(returns), + ) + + +def _quotient_modular_vpa( + target_cls: type[ModularVisiblyPushdownAutomaton], + source: ModularVisiblyPushdownAutomaton, + partition: Sequence[frozenset[Hashable]], + *, + modules: Mapping[Hashable, frozenset[Hashable]], + base_module: Hashable, + call_partition: Mapping[Any, Hashable], + call_entries: Mapping[Any, Hashable], + entry_states: Mapping[Hashable, Any], + form: str, +) -> Any: + block_of = _block_map(partition) + blocks = list(partition) + stack_alphabet: set[Any] = set() + if source.bottom_stack_symbol is not None: + stack_alphabet.add(source.bottom_stack_symbol) + transitions: set[tuple[Hashable, Hashable, Any, Any, Any | None]] = set() + stack_symbol_rewrite: dict[Any, set[Any]] = {} + + for transition in source.transitions(): + if transition.data.get(ATTR_KIND) != KIND_CALL: + continue + symbol = transition.data.get(ATTR_SYMBOL) + stack_symbol = transition.data.get(ATTR_STACK_SYMBOL) + quotient_stack_symbol = _quotient_call_stack_symbol( + form, + block_of[transition.source], + symbol, + stack_symbol, + block_of, + ) + stack_symbol_rewrite.setdefault(stack_symbol, set()).add(quotient_stack_symbol) + + for block in blocks: + representative = min(block, key=repr) + source_block = block_of[representative] + for transition in source.graph.out_transitions(representative): + kind = transition.data.get(ATTR_KIND) + symbol = transition.data.get(ATTR_SYMBOL) + target_block = block_of[transition.target] + stack_symbol = transition.data.get(ATTR_STACK_SYMBOL) + if kind == KIND_CALL: + stack_symbol = _quotient_call_stack_symbol(form, source_block, symbol, stack_symbol, block_of) + stack_alphabet.add(stack_symbol) + elif kind == KIND_RETURN and stack_symbol is not None: + rewritten = stack_symbol_rewrite.get(stack_symbol, {_canonical_stack_symbol(stack_symbol, block_of)}) + for rewritten_stack_symbol in rewritten: + stack_alphabet.add(rewritten_stack_symbol) + transitions.add((source_block, target_block, kind, symbol, rewritten_stack_symbol)) + continue + transitions.add((source_block, target_block, kind, symbol, stack_symbol)) + + quotient_modules = { + module: frozenset(block for block in blocks if block & states) + for module, states in modules.items() + if any(block & states for block in blocks) + } + quotient_call_entries = {symbol: block_of[state] for symbol, state in call_entries.items() if state in block_of} + quotient_entry_states = _quotient_entry_states(form, entry_states, block_of) + + kwargs = { + "input_alphabet": source.input_alphabet, + "call_alphabet": source.call_alphabet, + "return_alphabet": source.return_alphabet, + "internal_alphabet": source.internal_alphabet, + "stack_alphabet": frozenset(stack_alphabet), + "bottom_stack_symbol": source.bottom_stack_symbol, + "initial_state": block_of[source.initial_state], + "accepting_states": frozenset(block for block in blocks if block & source.accepting_states), + "modules": quotient_modules, + "base_module": base_module, + "call_partition": dict(call_partition), + "call_entries": quotient_call_entries, + } + if issubclass(target_cls, (SingleEntryVisiblyPushdownAutomaton, MultipleEntryVisiblyPushdownAutomaton)): + kwargs["entry_states"] = quotient_entry_states + + result = target_cls(**kwargs) + for block in blocks: + result.graph.add_state(block) + for source_block, target_block, kind, symbol, stack_symbol in sorted(transitions, key=repr): + if kind == KIND_CALL: + result.add_call_transition(source_block, target_block, symbol, stack_symbol) + elif kind == KIND_RETURN: + result.add_return_transition(source_block, target_block, symbol, stack_symbol) + elif kind == KIND_INTERNAL: + result.add_internal_transition(source_block, target_block, symbol) + result.validate() + return result + + +def _quotient_call_stack_symbol( + form: str, + source_block: Hashable, + symbol: Any, + stack_symbol: Any, + block_of: Mapping[Hashable, Hashable], +) -> Any: + if form == "sevpa": + return (source_block, symbol) + if form == "mevpa": + return source_block + return _canonical_stack_symbol(stack_symbol, block_of) + + +def _quotient_entry_states( + form: str, + entry_states: Mapping[Hashable, Any], + block_of: Mapping[Hashable, Hashable], +) -> dict[Hashable, Any]: + if form == "mevpa": + normalized = _normalize_multi_entries(entry_states) + return { + module: frozenset(block_of[state] for state in states if state in block_of) + for module, states in normalized.items() + } + if form == "sevpa": + return {module: block_of[state] for module, state in entry_states.items() if state in block_of} + return {} + + +def _canonical_stack_symbol(stack_symbol: Any, block_of: Mapping[Hashable, Hashable]) -> Any: + if stack_symbol in block_of: + return block_of[stack_symbol] + if isinstance(stack_symbol, tuple): + return tuple(_canonical_stack_symbol(part, block_of) for part in stack_symbol) + return stack_symbol diff --git a/sofic/automata/vpa/operations.py b/sofic/automata/vpa/operations.py new file mode 100644 index 0000000..1a46dae --- /dev/null +++ b/sofic/automata/vpa/operations.py @@ -0,0 +1,715 @@ +r"""Concrete constructions and decision procedures for visibly pushdown automata. + +Every construction works on a *normalized* form of a VPA: the bottom of the +stack is an explicit symbol ``⊥``, and wildcard returns are expanded into +guarded returns. A return guarded by ``⊥`` is a *pending* return: it fires on +the empty stack and leaves it empty. A VPA without ``⊥``-guarded returns rejects +pending returns. Acceptance is by final state, whatever the stack holds, so +pending calls are allowed :cite:`AlurMadhusudan2009`. + +* :func:`determinize` is the summary construction of Alur and Madhusudan: a + deterministic state is a pair ``(S, R)`` where ``S`` relates the state after + the last pending call to the current state along well-matched factors, and + ``R`` is the set of current states. The result is complete, so + :func:`complement` flips its accepting states. +* :func:`concat` and :func:`kleene_star` track, in the finite control, whether + the current factor's stack is empty, and push that bit with every symbol. A + return seen while the bit is set is a pending return of the current factor, + whatever lies below on the physical stack. +* :func:`is_empty` saturates the relation of well-matched summaries and then + explores states reachable with pending returns (stack empty) or pending calls. +* :func:`to_single_entry` and :func:`to_multiple_entry` convert a VPA of a + well-matched language into a modular form following the summary construction + of :cite:`AlurKumarMadhusudanViswanathan2005`: on a call the state resets to + the entry of the call's module and the caller is pushed. +""" + +from __future__ import annotations + +import heapq +from collections import defaultdict +from collections.abc import Hashable, Mapping +from dataclasses import dataclass, field +from typing import TYPE_CHECKING, Any + +from sofic.exceptions import NonWellMatchedLanguageError +from sofic.graph import ATTR_KIND, ATTR_STACK_SYMBOL, ATTR_SYMBOL, KIND_CALL, KIND_INTERNAL, KIND_RETURN + +if TYPE_CHECKING: + from sofic.automata.vpa.base import VisiblyPushdownAutomaton + from sofic.automata.vpa.deterministic import DeterministicVisiblyPushdownAutomaton + from sofic.automata.vpa.modular import MultipleEntryVisiblyPushdownAutomaton, SingleEntryVisiblyPushdownAutomaton + +#: Bottom-of-stack symbol used by constructed VPAs. +BOTTOM = "⊥" + +Word = tuple[Any, ...] + + +@dataclass +class NormalVPA: + """VPA with an explicit bottom symbol and only guarded returns.""" + + call_alphabet: frozenset[Any] + return_alphabet: frozenset[Any] + internal_alphabet: frozenset[Any] + states: set[Hashable] = field(default_factory=set) + initial: set[Hashable] = field(default_factory=set) + accepting: set[Hashable] = field(default_factory=set) + calls: set[tuple[Hashable, Any, Hashable, Any]] = field(default_factory=set) + internals: set[tuple[Hashable, Any, Hashable]] = field(default_factory=set) + returns: set[tuple[Hashable, Any, Any, Hashable]] = field(default_factory=set) + + @property + def stack_symbols(self) -> set[Any]: + pushed = {g for _q, _a, _t, g in self.calls} + guarded = {g for _q, _a, g, _t in self.returns if g != BOTTOM} + return pushed | guarded + + def call_map(self) -> dict[tuple[Hashable, Any], set[tuple[Hashable, Any]]]: + result: dict[tuple[Hashable, Any], set[tuple[Hashable, Any]]] = defaultdict(set) + for q, a, t, g in self.calls: + result[(q, a)].add((t, g)) + return result + + def internal_map(self) -> dict[tuple[Hashable, Any], set[Hashable]]: + result: dict[tuple[Hashable, Any], set[Hashable]] = defaultdict(set) + for q, a, t in self.internals: + result[(q, a)].add(t) + return result + + def return_map(self) -> dict[tuple[Hashable, Any], set[tuple[Any, Hashable]]]: + result: dict[tuple[Hashable, Any], set[tuple[Any, Hashable]]] = defaultdict(set) + for q, a, g, t in self.returns: + result[(q, a)].add((g, t)) + return result + + def copy_outgoing(self, source: Hashable, new_source: Hashable, map_target=lambda t: t) -> None: + """Give ``new_source`` copies of ``source``'s outgoing transitions.""" + self.calls |= {(new_source, a, map_target(t), g) for q, a, t, g in list(self.calls) if q == source} + self.internals |= {(new_source, a, map_target(t)) for q, a, t in list(self.internals) if q == source} + self.returns |= {(new_source, a, g, map_target(t)) for q, a, g, t in list(self.returns) if q == source} + + +def normalize(vpa: VisiblyPushdownAutomaton) -> NormalVPA: + """Return the normalized form of ``vpa``. + + A wildcard return (no stack symbol) fires on every stack symbol, and also on + the empty stack when ``vpa`` has a bottom symbol; a return guarded by the + bottom symbol fires only on the empty stack. A call without a stack symbol + never fires and is dropped. + """ + bottom = vpa.bottom_stack_symbol + gamma = {g for g in vpa.stack_alphabet if g != bottom} + for transition in vpa.transitions(): + if transition.data.get(ATTR_KIND) == KIND_CALL and transition.data.get(ATTR_STACK_SYMBOL) is not None: + gamma.add(transition.data.get(ATTR_STACK_SYMBOL)) + result = NormalVPA( + call_alphabet=frozenset(vpa.call_alphabet), + return_alphabet=frozenset(vpa.return_alphabet), + internal_alphabet=frozenset(vpa.internal_alphabet), + states=set(vpa.states()), + initial=set() if vpa.initial_state is None else {vpa.initial_state}, + accepting=set(vpa.accepting_states), + ) + for transition in vpa.transitions(): + data = transition.data + kind, symbol = data.get(ATTR_KIND), data.get(ATTR_SYMBOL) + if symbol is None: + continue + source, target = transition.source, transition.target + if kind == KIND_CALL: + if data.get(ATTR_STACK_SYMBOL) is not None: + result.calls.add((source, symbol, target, data.get(ATTR_STACK_SYMBOL))) + elif kind == KIND_INTERNAL: + result.internals.add((source, symbol, target)) + elif kind == KIND_RETURN: + guard = data.get(ATTR_STACK_SYMBOL) + if guard is None: + guards = set(gamma) | ({BOTTOM} if bottom is not None else set()) + elif bottom is not None and guard == bottom: + guards = {BOTTOM} + else: + guards = {guard} + result.returns |= {(source, symbol, g, target) for g in guards} + return result + + +def _merge_alphabets(*machines: NormalVPA) -> tuple[frozenset[Any], frozenset[Any], frozenset[Any]]: + calls = frozenset().union(*(m.call_alphabet for m in machines)) + returns = frozenset().union(*(m.return_alphabet for m in machines)) + internals = frozenset().union(*(m.internal_alphabet for m in machines)) + if len(calls) + len(returns) + len(internals) != len(calls | returns | internals): + raise ValueError("operands disagree on which symbols are calls, returns, and internals") + return calls, returns, internals + + +def _with_alphabets(machine: NormalVPA, calls, returns, internals) -> NormalVPA: + machine.call_alphabet, machine.return_alphabet, machine.internal_alphabet = calls, returns, internals + return machine + + +def _forward_trim(machine: NormalVPA) -> NormalVPA: + """Drop states unreachable from the initial states (ignoring the stack).""" + successors: dict[Hashable, set[Hashable]] = defaultdict(set) + for q, _a, t, _g in machine.calls: + successors[q].add(t) + for q, _a, t in machine.internals: + successors[q].add(t) + for q, _a, _g, t in machine.returns: + successors[q].add(t) + keep = set(machine.initial) + stack = list(keep) + while stack: + for target in successors[stack.pop()]: + if target not in keep: + keep.add(target) + stack.append(target) + machine.states &= keep + machine.accepting &= keep + machine.calls = {c for c in machine.calls if c[0] in keep} + machine.internals = {i for i in machine.internals if i[0] in keep} + machine.returns = {r for r in machine.returns if r[0] in keep} + return machine + + +def denormalize(machine: NormalVPA, cls: type | None = None, **extra: Any) -> Any: + """Build a VPA of class ``cls`` from a normalized form, relabeling states and stack symbols as integers.""" + from sofic.automata.vpa.base import VisiblyPushdownAutomaton + + cls = cls or VisiblyPushdownAutomaton + machine = _forward_trim(machine) + if len(machine.initial) > 1: + start = ("start", len(machine.states)) + machine.states.add(start) + for state in sorted(machine.initial, key=repr): + machine.copy_outgoing(state, start) + if machine.initial & machine.accepting: + machine.accepting.add(start) + machine.initial = {start} + + state_name = {state: index for index, state in enumerate(sorted(machine.states, key=repr))} + stack_name = {symbol: index for index, symbol in enumerate(sorted(machine.stack_symbols, key=repr))} + has_bottom = any(g == BOTTOM for _q, _a, g, _t in machine.returns) + stack_alphabet = set(stack_name.values()) | ({BOTTOM} if has_bottom else set()) + result = cls( + call_alphabet=machine.call_alphabet, + return_alphabet=machine.return_alphabet, + internal_alphabet=machine.internal_alphabet, + stack_alphabet=frozenset(stack_alphabet), + bottom_stack_symbol=BOTTOM if has_bottom else None, + initial_state=state_name[next(iter(machine.initial))] if machine.initial else None, + accepting_states=frozenset(state_name[q] for q in machine.accepting), + **extra, + ) + for state in state_name.values(): + result.graph.add_state(state) + for q, a, t, g in sorted(machine.calls, key=repr): + result.graph.add_transition( + state_name[q], state_name[t], **{ATTR_KIND: KIND_CALL, ATTR_SYMBOL: a, ATTR_STACK_SYMBOL: stack_name[g]} + ) + for q, a, t in sorted(machine.internals, key=repr): + result.graph.add_transition(state_name[q], state_name[t], **{ATTR_KIND: KIND_INTERNAL, ATTR_SYMBOL: a}) + for q, a, g, t in sorted(machine.returns, key=repr): + guard = BOTTOM if g == BOTTOM else stack_name[g] + result.graph.add_transition( + state_name[q], state_name[t], **{ATTR_KIND: KIND_RETURN, ATTR_SYMBOL: a, ATTR_STACK_SYMBOL: guard} + ) + return result + + +# --------------------------------------------------------------------- boolean operations + + +def union(left: NormalVPA, right: NormalVPA) -> NormalVPA: + """Disjoint sum of two normalized VPAs.""" + alphabets = _merge_alphabets(left, right) + result = NormalVPA(*alphabets) + for tag, machine in ((0, left), (1, right)): + + def lift(g: Any, tag: int = tag) -> Any: + return BOTTOM if g == BOTTOM else (tag, g) + + result.states |= {(tag, q) for q in machine.states} + result.initial |= {(tag, q) for q in machine.initial} + result.accepting |= {(tag, q) for q in machine.accepting} + result.calls |= {((tag, q), a, (tag, t), (tag, g)) for q, a, t, g in machine.calls} + result.internals |= {((tag, q), a, (tag, t)) for q, a, t in machine.internals} + result.returns |= {((tag, q), a, lift(g), (tag, t)) for q, a, g, t in machine.returns} + return result + + +def intersection(left: NormalVPA, right: NormalVPA) -> NormalVPA: + """Synchronized product of two normalized VPAs (reachable part).""" + alphabets = _merge_alphabets(left, right) + result = NormalVPA(*alphabets) + lc, li, lr = left.call_map(), left.internal_map(), left.return_map() + rc, ri, rr = right.call_map(), right.internal_map(), right.return_map() + frontier = [(p, q) for p in left.initial for q in right.initial] + result.initial = set(frontier) + result.states = set(frontier) + while frontier: + state = frontier.pop() + p, q = state + if p in left.accepting and q in right.accepting: + result.accepting.add(state) + successors = [] + for a in alphabets[2]: + for t1 in li.get((p, a), ()): + for t2 in ri.get((q, a), ()): + result.internals.add((state, a, (t1, t2))) + successors.append((t1, t2)) + for a in alphabets[0]: + for t1, g1 in lc.get((p, a), ()): + for t2, g2 in rc.get((q, a), ()): + result.calls.add((state, a, (t1, t2), (g1, g2))) + successors.append((t1, t2)) + for a in alphabets[1]: + for g1, t1 in lr.get((p, a), ()): + for g2, t2 in rr.get((q, a), ()): + if (g1 == BOTTOM) != (g2 == BOTTOM): + continue + guard = BOTTOM if g1 == BOTTOM else (g1, g2) + result.returns.add((state, a, guard, (t1, t2))) + successors.append((t1, t2)) + for successor in successors: + if successor not in result.states: + result.states.add(successor) + frontier.append(successor) + return result + + +def determinize(machine: NormalVPA, alphabets: tuple[frozenset, frozenset, frozenset] | None = None) -> NormalVPA: + """Alur-Madhusudan summary construction; the result is deterministic and complete.""" + calls_a, returns_a, internals_a = alphabets or ( + machine.call_alphabet, + machine.return_alphabet, + machine.internal_alphabet, + ) + cmap, imap, rmap = machine.call_map(), machine.internal_map(), machine.return_map() + identity = frozenset((q, q) for q in machine.states) + start = (identity, frozenset(machine.initial)) + result = NormalVPA(calls_a, returns_a, internals_a, states={start}, initial={start}) + + def after_internal(pairs, a): + return frozenset((p, t) for p, q in pairs for t in imap.get((q, a), ())) + + def after_pending_return(pairs, a): + return frozenset((p, t) for p, q in pairs for g, t in rmap.get((q, a), ()) if g == BOTTOM) + + def after_matched_return(caller_pairs, c, pairs, a): + inner: dict[Hashable, set[Hashable]] = defaultdict(set) + for q2, q3 in pairs: + inner[q2].add(q3) + out = set() + for p, q1 in caller_pairs: + for q2, g in cmap.get((q1, c), ()): + for q3 in inner.get(q2, ()): + out |= {(p, t) for guard, t in rmap.get((q3, a), ()) if guard == g} + return frozenset(out) + + stack_symbols: list[tuple[frozenset, frozenset, Any]] = [] + known_stack: set[tuple[frozenset, frozenset, Any]] = set() + done: set[tuple[Any, Any]] = set() + pending = [start] + while True: + while pending: + state = pending.pop() + pairs, current = state + if current & machine.accepting: + result.accepting.add(state) + successors = [] + for a in internals_a: + target = (after_internal(pairs, a), frozenset(t for q in current for t in imap.get((q, a), ()))) + successors.append(target) + result.internals.add((state, a, target)) + for c in calls_a: + entered = frozenset(t for q in current for t, _g in cmap.get((q, c), ())) + target = (identity, entered) + symbol = (pairs, current, c) + if symbol not in known_stack: + known_stack.add(symbol) + stack_symbols.append(symbol) + successors.append(target) + result.calls.add((state, c, target, symbol)) + for a in returns_a: + moved = after_pending_return(pairs, a) + current_after = frozenset(t for q in current for g, t in rmap.get((q, a), ()) if g == BOTTOM) + target = (moved, current_after) + successors.append(target) + result.returns.add((state, a, BOTTOM, target)) + for successor in successors: + if successor not in result.states: + result.states.add(successor) + pending.append(successor) + new_work = False + for state in list(result.states): + pairs, _current = state + for symbol in list(stack_symbols): + if (state, symbol) in done: + continue + done.add((state, symbol)) + caller_pairs, caller_current, c = symbol + for a in returns_a: + moved = after_matched_return(caller_pairs, c, pairs, a) + reached = after_matched_return(frozenset((q, q) for q in caller_current), c, pairs, a) + target = (moved, frozenset(t for _p, t in reached)) + result.returns.add((state, a, symbol, target)) + if target not in result.states: + result.states.add(target) + pending.append(target) + new_work = True + if not new_work and not pending: + return result + + +def complement(machine: NormalVPA, alphabets=None) -> NormalVPA: + """Complement over the visible alphabet (or ``alphabets``).""" + deterministic = determinize(machine, alphabets) + deterministic.accepting = deterministic.states - deterministic.accepting + return deterministic + + +def difference(left: NormalVPA, right: NormalVPA) -> NormalVPA: + alphabets = _merge_alphabets(left, right) + return intersection(_with_alphabets(left, *alphabets), complement(right, alphabets)) + + +# ----------------------------------------------------------------- structural operations + + +def concat(left: NormalVPA, right: NormalVPA) -> NormalVPA: + """Concatenation; the right factor starts with an empty stack of its own.""" + alphabets = _merge_alphabets(left, right) + result = NormalVPA(*alphabets) + left_tops = [("A", g) for g in left.stack_symbols] + [BOTTOM] + + for q in left.states: + result.states.add(("A", q)) + result.initial = {("A", q) for q in left.initial} + result.calls |= {(("A", q), a, ("A", t), ("A", g)) for q, a, t, g in left.calls} + result.internals |= {(("A", q), a, ("A", t)) for q, a, t in left.internals} + result.returns |= {(("A", q), a, g if g == BOTTOM else ("A", g), ("A", t)) for q, a, g, t in left.returns} + + for q in right.states: + for empty in (True, False): + state = ("B", q, empty) + result.states.add(state) + result.internals |= {(state, a, ("B", t, empty)) for p, a, t in right.internals if p == q} + result.calls |= {(state, a, ("B", t, False), ("B", g, empty)) for p, a, t, g in right.calls if p == q} + for p, a, g, t in right.returns: + if p != q: + continue + if empty and g == BOTTOM: + result.returns |= {(state, a, top, ("B", t, True)) for top in left_tops} + elif not empty and g != BOTTOM: + result.returns |= {(state, a, ("B", g, bit), ("B", t, bit)) for bit in (True, False)} + if q in right.accepting: + result.accepting.add(state) + + epsilon_in_right = bool(right.initial & right.accepting) + for q in left.accepting: + for start in right.initial: + result.copy_outgoing(("B", start, True), ("A", q)) + if epsilon_in_right: + result.accepting.add(("A", q)) + return result + + +def kleene_star(machine: NormalVPA) -> NormalVPA: + """Kleene star; each factor starts with an empty stack of its own.""" + result = NormalVPA(machine.call_alphabet, machine.return_alphabet, machine.internal_alphabet) + tops = [(g, bit) for g in machine.stack_symbols for bit in (True, False)] + [BOTTOM] + for q in machine.states: + for empty in (True, False): + state = (q, empty) + result.states.add(state) + result.internals |= {(state, a, (t, empty)) for p, a, t in machine.internals if p == q} + result.calls |= {(state, a, (t, False), (g, empty)) for p, a, t, g in machine.calls if p == q} + for p, a, g, t in machine.returns: + if p != q: + continue + if empty and g == BOTTOM: + result.returns |= {(state, a, top, (t, True)) for top in tops} + elif not empty and g != BOTTOM: + result.returns |= {(state, a, (g, bit), (t, bit)) for bit in (True, False)} + if q in machine.accepting: + result.accepting.add(state) + start = ("star-start",) + result.states.add(start) + result.initial = {start} + result.accepting.add(start) + restart_sources = [start] + [(q, empty) for q in machine.accepting for empty in (True, False)] + for source in restart_sources: + for initial in machine.initial: + result.copy_outgoing((initial, True), source) + return result + + +# --------------------------------------------------------------------- decision procedures + + +def well_matched_summaries(machine: NormalVPA) -> dict[tuple[Hashable, Hashable], Word]: + """Pairs ``(p, q)`` joined by a well-matched word, each with a witness word.""" + witness: dict[tuple[Hashable, Hashable], Word] = {(q, q): () for q in machine.states} + rmap = machine.return_map() + changed = True + while changed: + changed = False + updates: dict[tuple[Hashable, Hashable], Word] = {} + by_source: dict[Hashable, list[tuple[Hashable, Word]]] = defaultdict(list) + for (p, q), word in witness.items(): + by_source[p].append((q, word)) + for (p, q), word in witness.items(): + for source, a, target in machine.internals: + if source == q: + updates.setdefault((p, target), (*word, a)) + for r, word2 in by_source.get(q, ()): + updates.setdefault((p, r), word + word2) + for p0, c, p, g in machine.calls: + for q, word in by_source.get(p, ()): + for a in machine.return_alphabet: + for guard, target in rmap.get((q, a), ()): + if guard == g: + updates.setdefault((p0, target), (c, *word, a)) + for pair, word in updates.items(): + if pair not in witness or len(word) < len(witness[pair]): + if pair not in witness: + changed = True + witness[pair] = word + return witness + + +def accepted_word(machine: NormalVPA) -> Word | None: + """Return a short accepted word, or ``None`` when the language is empty.""" + summaries = well_matched_summaries(machine) + by_source: dict[Hashable, list[tuple[Hashable, Word]]] = defaultdict(list) + for (p, q), word in summaries.items(): + if word: + by_source[p].append((q, word)) + queue: list[tuple[int, int, Hashable, bool, Word]] = [] + counter = 0 + for q in machine.initial: + heapq.heappush(queue, (0, counter, q, True, ())) + counter += 1 + seen: set[tuple[Hashable, bool]] = set() + while queue: + _length, _tie, state, stack_empty, word = heapq.heappop(queue) + if (state, stack_empty) in seen: + continue + seen.add((state, stack_empty)) + if state in machine.accepting: + return word + moves: list[tuple[Hashable, bool, Word]] = [(t, stack_empty, word + w) for t, w in by_source.get(state, ())] + moves += [(t, False, (*word, a)) for q, a, t, _g in machine.calls if q == state] + if stack_empty: + moves += [(t, True, (*word, a)) for q, a, g, t in machine.returns if q == state and g == BOTTOM] + for target, empty, extended in moves: + if (target, empty) not in seen: + heapq.heappush(queue, (len(extended), counter, target, empty, extended)) + counter += 1 + return None + + +def is_empty(machine: NormalVPA) -> bool: + return accepted_word(machine) is None + + +def _unmatched_tracker(alphabets: tuple[frozenset, frozenset, frozenset]) -> NormalVPA: + """Deterministic VPA accepting the words with a pending call or a pending return.""" + tracker = NormalVPA(*alphabets) + for pending_return in (False, True): + for empty in (True, False): + state = (pending_return, empty) + tracker.states.add(state) + if pending_return or not empty: + tracker.accepting.add(state) + tracker.internals |= {(state, a, state) for a in alphabets[2]} + tracker.calls |= {(state, c, (pending_return, False), empty) for c in alphabets[0]} + for a in alphabets[1]: + if empty: + tracker.returns.add((state, a, BOTTOM, (True, True))) + else: + tracker.returns |= {(state, a, bit, (pending_return, bit)) for bit in (True, False)} + tracker.initial = {(False, True)} + return tracker + + +def has_unmatched_word(machine: NormalVPA) -> bool: + """Whether the language contains a word with a pending call or a pending return.""" + alphabets = (machine.call_alphabet, machine.return_alphabet, machine.internal_alphabet) + return not is_empty(intersection(machine, _unmatched_tracker(alphabets))) + + +# ------------------------------------------------------------------------ modular forms + + +def _modular_conversion( + vpa: VisiblyPushdownAutomaton, + call_partition: Mapping[Any, Hashable] | None, + *, + multiple_entry: bool, +) -> Any: + from sofic.automata.vpa.modular import MultipleEntryVisiblyPushdownAutomaton, SingleEntryVisiblyPushdownAutomaton + + machine = normalize(vpa) + if has_unmatched_word(machine): + raise NonWellMatchedLanguageError( + "modular (single- or multiple-entry) forms are built here for well-matched languages only" + ) + base = "base" + partition = dict(call_partition) if call_partition is not None else {c: ("module", c) for c in vpa.call_alphabet} + if base in partition.values(): + raise ValueError("calls cannot target the base module") + missing = set(vpa.call_alphabet) - set(partition) + if missing: + raise ValueError(f"call_partition is missing calls {sorted(missing, key=repr)!r}") + + cmap, imap, rmap = machine.call_map(), machine.internal_map(), machine.return_map() + identity = frozenset((q, q) for q in machine.states) + + def compose_internal(pairs, a): + return frozenset((p, t) for p, q in pairs for t in imap.get((q, a), ())) + + def wrap(caller_pairs, c, pairs, a): + inner: dict[Hashable, set[Hashable]] = defaultdict(set) + for q2, q3 in pairs: + inner[q2].add(q3) + out = set() + for p, q1 in caller_pairs: + for q2, g in cmap.get((q1, c), ()): + for q3 in inner.get(q2, ()): + out |= {(p, t) for guard, t in rmap.get((q3, a), ()) if guard == g} + return frozenset(out) + + # A raw state is (module, entering_call, summary); entering_call is only kept + # for the multiple-entry form, where it replaces the call symbol on the stack. + def entry(c): + return (partition[c], c if multiple_entry else None, identity) + + start = (base, None, identity) + label: dict[tuple, int] = {start: 0} + order = [start] + internals: set[tuple[int, Any, int]] = set() + calls: set[tuple[int, Any, int, Any]] = set() + returns: set[tuple[int, Any, Any, int]] = set() + pushes: list[tuple[int, Any]] = [] + done: set[tuple[int, Any]] = set() + + def name(raw): + if raw not in label: + label[raw] = len(order) + order.append(raw) + return label[raw] + + index = 0 + while True: + while index < len(order): + raw = order[index] + module, entered, pairs = raw + source = label[raw] + for a in vpa.internal_alphabet: + internals.add((source, a, name((module, entered, compose_internal(pairs, a))))) + for c in vpa.call_alphabet: + push = source if multiple_entry else (source, c) + calls.add((source, c, name(entry(c)), push)) + if (source, c) not in pushes: + pushes.append((source, c)) + index += 1 + grew = False + for raw in list(order): + module, entered, pairs = raw + source = label[raw] + for caller, c in list(pushes): + if (source, (caller, c)) in done: + continue + if multiple_entry and entered != c: + continue + if not multiple_entry and module != partition[c]: + continue + done.add((source, (caller, c))) + caller_module, caller_entered, caller_pairs = order[caller] + guard = caller if multiple_entry else (caller, c) + for a in vpa.return_alphabet: + before = len(order) + target = name((caller_module, caller_entered, wrap(caller_pairs, c, pairs, a))) + returns.add((source, a, guard, target)) + grew = grew or len(order) > before + if not grew and index >= len(order): + break + + modules: dict[Hashable, set[int]] = defaultdict(set) + for raw, state in label.items(): + modules[raw[0]].add(state) + accepting = { + label[raw] + for raw in order + if raw[0] == base and any(p in machine.initial and q in machine.accepting for p, q in raw[2]) + } + used_partition = {c: m for c, m in partition.items() if c in vpa.call_alphabet} + call_entries = {c: label[entry(c)] for c in vpa.call_alphabet} + common = { + "call_alphabet": vpa.call_alphabet, + "return_alphabet": vpa.return_alphabet, + "internal_alphabet": vpa.internal_alphabet, + "stack_alphabet": frozenset(push for _s, _c, _t, push in calls), + "initial_state": 0, + "accepting_states": frozenset(accepting), + "modules": {m: frozenset(states) for m, states in modules.items()}, + "base_module": base, + "call_partition": used_partition, + "call_entries": call_entries, + } + if multiple_entry: + entries: dict[Hashable, set[int]] = defaultdict(set) + entries[base].add(0) + for c, state in call_entries.items(): + entries[partition[c]].add(state) + result = MultipleEntryVisiblyPushdownAutomaton( + entry_states={m: frozenset(s) for m, s in entries.items()}, **common + ) + else: + result = SingleEntryVisiblyPushdownAutomaton( + entry_states={partition[c]: state for c, state in call_entries.items()}, **common + ) + for state in range(len(order)): + result.graph.add_state(state) + for s, c, t, push in sorted(calls, key=repr): + result.add_call_transition(s, t, c, push) + for s, a, t in sorted(internals, key=repr): + result.add_internal_transition(s, t, a) + for s, a, guard, t in sorted(returns, key=repr): + result.add_return_transition(s, t, a, guard) + result.validate() + return result + + +def to_single_entry( + vpa: VisiblyPushdownAutomaton, call_partition: Mapping[Any, Hashable] | None = None +) -> SingleEntryVisiblyPushdownAutomaton: + """Convert a VPA of a well-matched language into a single-entry modular VPA. + + ``call_partition`` maps each call symbol to a module (default: one module + per call symbol). Raises + :class:`~sofic.exceptions.NonWellMatchedLanguageError` when the language has + a word with a pending call or return, which a single-entry automaton that + forgets its caller cannot accept. + """ + return _modular_conversion(vpa, call_partition, multiple_entry=False) + + +def to_multiple_entry( + vpa: VisiblyPushdownAutomaton, call_partition: Mapping[Any, Hashable] | None = None +) -> MultipleEntryVisiblyPushdownAutomaton: + """Convert a VPA of a well-matched language into a multiple-entry modular VPA. + + Each module has one entry per call symbol assigned to it, and calls push + only the caller state. + """ + return _modular_conversion(vpa, call_partition, multiple_entry=True) + + +def determinize_vpa(vpa: VisiblyPushdownAutomaton) -> DeterministicVisiblyPushdownAutomaton: + from sofic.automata.vpa.deterministic import DeterministicVisiblyPushdownAutomaton + + return denormalize(determinize(normalize(vpa)), DeterministicVisiblyPushdownAutomaton) diff --git a/sofic/automata/vpa/simulation.py b/sofic/automata/vpa/simulation.py new file mode 100644 index 0000000..b0bc2d9 --- /dev/null +++ b/sofic/automata/vpa/simulation.py @@ -0,0 +1,40 @@ +"""Stack simulation for visibly pushdown automata.""" + +from __future__ import annotations + +from collections.abc import Hashable, Iterator, Sequence +from typing import Any + +from sofic.automata._config_simulation import simulate_configs +from sofic.automata.vpa.base import VisiblyPushdownAutomaton +from sofic.automata.vpa.operations import BOTTOM, normalize + +Config = tuple[Hashable, tuple[Any, ...]] + + +def recognizes_vpa(vpa: VisiblyPushdownAutomaton, word: Sequence[Any]) -> bool: + """Return whether ``vpa`` accepts ``word``. + + Simulates the normalized form (:func:`~sofic.automata.vpa.operations.normalize`): + the stack holds only pushed symbols, its emptiness stands for the explicit + bottom, and a return guarded by ``BOTTOM`` fires only on the empty stack. + """ + machine = normalize(vpa) + calls, internals, returns = machine.call_map(), machine.internal_map(), machine.return_map() + + def step(config: Config, symbol: Any) -> Iterator[Config]: + state, stack = config + for target, pushed in calls.get((state, symbol), ()): + yield (target, (*stack, pushed)) + for target in internals.get((state, symbol), ()): + yield (target, stack) + for guard, target in returns.get((state, symbol), ()): + if guard == BOTTOM: + if not stack: + yield (target, stack) + elif stack and stack[-1] == guard: + yield (target, stack[:-1]) + + initial: set[Config] = {(state, ()) for state in machine.initial} + current = simulate_configs(initial, word, step) + return any(state in machine.accepting for state, _ in current) diff --git a/sofic/automata/vpa_simulation.py b/sofic/automata/vpa_simulation.py deleted file mode 100644 index 55efe6c..0000000 --- a/sofic/automata/vpa_simulation.py +++ /dev/null @@ -1,53 +0,0 @@ -"""Stack simulation for visibly pushdown automata.""" - -from __future__ import annotations - -from collections.abc import Hashable, Iterator, Sequence -from typing import Any - -from sofic.automata._config_simulation import simulate_configs -from sofic.automata.vpa import VisiblyPushdownAutomaton -from sofic.graph import ATTR_KIND, ATTR_STACK_SYMBOL, ATTR_SYMBOL, KIND_CALL, KIND_INTERNAL, KIND_RETURN - -_BOTTOM = object() - -Config = tuple[Hashable, tuple[Any, ...]] - - -def recognizes_vpa(vpa: VisiblyPushdownAutomaton, word: Sequence[Any]) -> bool: - """Return whether ``vpa`` accepts ``word``.""" - if vpa.initial_state is None: - return False - - bottom = _BOTTOM if vpa.bottom_stack_symbol is None else vpa.bottom_stack_symbol - initial: set[Config] = {(vpa.initial_state, (bottom,))} - - def step(config: Config, symbol: Any) -> Iterator[Config]: - state, stack = config - for transition in vpa.graph.out_transitions(state): - data = transition.data - if data.get(ATTR_SYMBOL) != symbol: - continue - kind = data.get(ATTR_KIND) - if kind == KIND_CALL: - stack_sym = data.get(ATTR_STACK_SYMBOL) - if stack_sym is None: - continue - yield (transition.target, stack + (stack_sym,)) - elif kind == KIND_RETURN: - stack_sym = data.get(ATTR_STACK_SYMBOL) - if len(stack) <= 1: - if vpa.bottom_stack_symbol is None: - continue - if stack_sym is not None and stack_sym != stack[-1]: - continue - yield (transition.target, stack) - continue - if stack_sym is not None and stack_sym != stack[-1]: - continue - yield (transition.target, stack[:-1]) - elif kind == KIND_INTERNAL: - yield (transition.target, stack) - - current = simulate_configs(initial, word, step) - return any(state in vpa.accepting_states for state, _ in current) diff --git a/sofic/automata/wheeler.py b/sofic/automata/wheeler.py index 45b28ee..e99717e 100644 --- a/sofic/automata/wheeler.py +++ b/sofic/automata/wheeler.py @@ -493,7 +493,7 @@ def _order_of(model: Any, symbol_key: Callable[[Any], Any] | None) -> tuple[Labe return graph, order -def minimum_wdfa( +def minimize_wheeler( dfa: Any, *, symbol_key: Callable[[Any], Any] | None = None, @@ -576,7 +576,7 @@ def step(automaton: Any, state: Hashable, symbol: Any) -> Hashable | None: return classes -def wnfa_to_wdfa( +def determinize_wheeler( nfa: Any, *, symbol_key: Callable[[Any], Any] | None = None, diff --git a/sofic/examples/_construction.py b/sofic/examples/_construction.py new file mode 100644 index 0000000..09f9cb6 --- /dev/null +++ b/sofic/examples/_construction.py @@ -0,0 +1,131 @@ +"""Private helpers shared by the example constructor modules.""" + +from __future__ import annotations + +from collections.abc import Hashable, Iterable, Mapping, Sequence +from typing import Any + +import numpy as np + +from sofic.generators.epsilon_machine import EpsilonMachine +from sofic.generators.mealy import MealyHMM +from sofic.graph import ATTR_EMISSION, ATTR_PROB + + +def _uniform_initial(states: Sequence[Hashable]) -> dict[Hashable, float]: + if not states: + return {} + mass = 1.0 / len(states) + return dict.fromkeys(states, mass) + + +def _stationary_initial(states: Sequence[Hashable], transition: Any) -> dict[Hashable, Any]: + """Stationary distribution of ``transition`` keyed by ``states``. + + Float matrices are row-normalized first and fall back to the uniform + distribution when a row is empty or the solve fails; exact (sympy) matrices + are solved as given and keep exact entries. + """ + from sofic.generators.prob import as_prob, has_symbolic + from sofic.generators.stationary import stationary_distribution_from_transition + + if not states: + return {} + matrix = np.asarray(transition) + if not (matrix.dtype == object and has_symbolic(matrix.ravel())): + matrix = np.asarray(matrix, dtype=float) + row_sums = matrix.sum(axis=1) + if np.any(row_sums <= 0.0): + return _uniform_initial(states) + matrix = matrix / row_sums[:, None] + try: + pi = stationary_distribution_from_transition(matrix) + except (ValueError, np.linalg.LinAlgError): + return _uniform_initial(states) + if pi.dtype == object or has_symbolic(pi.ravel()): + return {state: as_prob(pi[i]) for i, state in enumerate(states)} + return {state: float(pi[i]) for i, state in enumerate(states)} + + +def _normalize_edges( + edges: Sequence[tuple[Hashable, Hashable, Any, float]], +) -> list[tuple[Hashable, Hashable, Any, float]]: + row_totals: dict[Hashable, float] = {} + for source, _target, _symbol, prob in edges: + row_totals[source] = row_totals.get(source, 0.0) + float(prob) + normalized = [] + for source, target, symbol, prob in edges: + total = row_totals[source] + normalized.append((source, target, symbol, float(prob) / total if total else 0.0)) + return normalized + + +def _edge_machine( + edges: Iterable[tuple[Hashable, Hashable, Any, float]], + *, + machine_type: type[MealyHMM] = EpsilonMachine, + name: str | None = None, + initial_distribution: Mapping[Hashable, float] | None = None, + normalize: bool = True, + validate: bool = True, +) -> MealyHMM: + edge_list = list(edges) + if normalize: + edge_list = _normalize_edges(edge_list) + + states = list(dict.fromkeys([source for source, *_ in edge_list] + [target for _source, target, *_ in edge_list])) + symbols = frozenset(symbol for _source, _target, symbol, _prob in edge_list) + if initial_distribution is not None: + initial = dict(initial_distribution) + else: + index = {state: i for i, state in enumerate(states)} + transition = np.zeros((len(states), len(states)), dtype=float) + for source, target, _symbol, prob in edge_list: + transition[index[source], index[target]] += float(prob) + initial = _stationary_initial(states, transition) + + machine = machine_type(initial_distribution=initial, observation_alphabet=symbols) + if name is not None: + machine.name = name + for state in states: + machine.graph.add_state(state) + for source, target, symbol, prob in edge_list: + if prob > 0.0: + machine.graph.add_transition(source, target, **{ATTR_EMISSION: symbol, ATTR_PROB: float(prob)}) + if validate: + machine.validate() + return machine + + +def _relabel( + machine: MealyHMM, + *, + symbols: Mapping[Any, Any] | None = None, + states: Mapping[Hashable, Hashable] | None = None, + machine_type: type[MealyHMM] | None = None, + name: str | None = None, +) -> MealyHMM: + """Copy ``machine`` with emissions and states renamed; probabilities are kept as-is. + + A ``states`` mapping that covers every state also fixes the output state + order (the order of its values). + """ + symbol_map = dict(symbols or {}) + state_map = dict(states or {}) + old_states = list(machine.graph.states()) + if set(state_map) >= set(old_states): + old_states = [old for old in state_map if old in set(old_states)] + out = (machine_type or type(machine))( + initial_distribution={state_map.get(s, s): p for s, p in machine.initial_distribution.items()}, + observation_alphabet=frozenset(symbol_map.get(x, x) for x in machine.observation_alphabet), + ) + if name is not None: + out.name = name + for state in old_states: + out.graph.add_state(state_map.get(state, state), **dict(machine.graph.state_attrs(state))) + for t in machine.transitions(): + data = dict(t.data) + data[ATTR_EMISSION] = symbol_map.get(data[ATTR_EMISSION], data[ATTR_EMISSION]) + out.graph.add_transition(state_map.get(t.source, t.source), state_map.get(t.target, t.target), **data) + out.validate() + return out diff --git a/sofic/examples/epsilon_machines.py b/sofic/examples/epsilon_machines.py index b30e777..99d40d7 100644 --- a/sofic/examples/epsilon_machines.py +++ b/sofic/examples/epsilon_machines.py @@ -27,6 +27,8 @@ import numpy as np +from sofic.examples._construction import _edge_machine, _relabel, _stationary_initial +from sofic.exceptions import SoficError from sofic.generators.epsilon_machine import EpsilonMachine from sofic.generators.mealy import MealyHMM from sofic.graph import ATTR_EMISSION, ATTR_FUTURE_SYMBOL, ATTR_PROB, TransitionGraph @@ -34,20 +36,6 @@ from sofic.states import sequential_labels -def _stationary_distribution( - states: Sequence[Hashable], - symbol_matrices: Mapping[Any, np.ndarray], -) -> dict[Hashable, Any]: - from sofic.generators.prob import as_prob, has_symbolic - from sofic.generators.stationary import stationary_distribution_from_transition - - transition = sum(symbol_matrices.values()) - pi = stationary_distribution_from_transition(transition) - if pi.dtype == object or has_symbolic(pi.ravel()): - return {states[i]: as_prob(pi[i]) for i in range(len(states))} - return {states[i]: float(pi[i]) for i in range(len(states))} - - def from_symbol_matrices( states: Sequence[Hashable], symbols: Sequence[Any], @@ -75,7 +63,9 @@ def from_symbol_matrices( arrays[symbol] = arr pi = ( - dict(initial_distribution) if initial_distribution is not None else _stationary_distribution(state_list, arrays) + dict(initial_distribution) + if initial_distribution is not None + else _stationary_initial(state_list, sum(arrays.values())) ) eps = EpsilonMachine( initial_distribution=pi, @@ -105,8 +95,6 @@ def bernoulli(p: float = 0.5, *, symbols: tuple[Any, Any] = ("0", "1")) -> Epsil """Memoryless (Bernoulli) source with ``P(symbols[0]) = 1 - p``.""" if not 0.0 < p < 1.0: raise ValueError("p must be in (0, 1)") - from sofic.examples.processes import _edge_machine - zero, one = symbols state = sequential_labels(1)[0] return _edge_machine( @@ -149,8 +137,6 @@ def noisy_random_phase_slip() -> EpsilonMachine: emission noise at state ``D``. Prototype for block-convergence figures in *Anatomy of a Bit* :cite:`James2011`. """ - from sofic.examples.processes import _edge_machine - states = sequential_labels(5) a, b, c, d, e = states return _edge_machine( @@ -178,7 +164,9 @@ def golden_mean(p: float = 0.5) -> EpsilonMachine: see :func:`golden_mean_markov`; for the bidirectional machine in Ellison et al., arXiv:0905.3587, Fig.~4, see :func:`golden_mean_forward` and :func:`golden_mean_reverse`; for the Parry max-entropy measure on the - same shift, see :func:`golden_mean_shift_parry`. + same shift, see :func:`golden_mean_shift_parry`. cmpy's + :func:`~sofic.examples.processes.golden_mean_forbid_00` is the ``0 <-> 1`` mirror + (forbids ``00``) and equals :func:`golden_mean_forward`. """ if not 0.0 < p < 1.0: raise ValueError("p must be in (0, 1)") @@ -243,16 +231,7 @@ def golden_mean_markov(p: float = 0.5) -> EpsilonMachine: For the paper's ``A``/``B`` labeling and bidirectional machine, prefer :func:`golden_mean_forward`. """ - if not 0.0 < p < 1.0: - raise ValueError("p must be in (0, 1)") - return from_symbol_matrices( - sequential_labels(2), - (0, 1), - { - 0: np.array([[0.0, 0.0], [p, 0.0]]), - 1: np.array([[0.0, 1.0], [0.0, 1.0 - p]]), - }, - ) + return _relabel(golden_mean_forward(1 - p), states={"B": "A", "A": "B"}) def golden_mean_shift_parry() -> EpsilonMachine: @@ -294,16 +273,10 @@ def restricted_golden_mean(k: int = 1) -> EpsilonMachine: """ if k < 1: raise ValueError("k must be >= 1") - states = sequential_labels(k + 1) - n = k + 1 - t0 = np.zeros((n, n), dtype=float) - t1 = np.zeros((n, n), dtype=float) - t0[0, 1] = 0.5 - t1[0, 0] = 0.5 - for i in range(1, k): - t1[i, i + 1] = 1.0 - t1[k, 0] = 1.0 - return from_symbol_matrices(states, (0, 1), {0: t0, 1: t1}) + from sofic.examples.processes import restricted_gm + + states = {str(i): label for i, label in enumerate(sequential_labels(k + 1))} + return _relabel(restricted_gm(k), symbols={"0": 0, "1": 1}, states=states) def nemo_process(p: float = 0.5, q: float = 0.5) -> EpsilonMachine: @@ -339,7 +312,8 @@ def nemo_process(p: float = 0.5, q: float = 0.5) -> EpsilonMachine: def phase_slip_backtrack(p: float = 0.5, q: float = 0.5) -> EpsilonMachine: """Phase-Slip Backtrack (PSB) Process (``R=3``, ``k_chi=2``). - James, Mahoney, Ellison & Crutchfield, arXiv:1010.5545, Fig.~2. + James, Mahoney, Ellison & Crutchfield, arXiv:1010.5545, Fig.~2. A different + process from cmpy's :func:`~sofic.examples.processes.phase_slip_backtrack_cmpy`. """ if not 0.0 < p < 1.0 or not 0.0 < q < 1.0: raise ValueError("p and q must be in (0, 1)") @@ -373,7 +347,8 @@ def butterfly_process() -> EpsilonMachine: Mahoney et al., arXiv:0906.5099, Fig.~1. Each causal state emits every symbol with probability ``1/8``; synchronizing symbols ``2``--``7`` always - reach the same causal state regardless of the source. + reach the same causal state regardless of the source. Not cmpy's + :func:`~sofic.examples.processes.butterfly_two_branch` (two branches per state, ``h_mu = 1``). """ states = ("A", "B", "C", "D", "E") prob = 1.0 / 8.0 @@ -387,8 +362,6 @@ def butterfly_process() -> EpsilonMachine: 6: "C", 7: "E", } - from sofic.examples.processes import _edge_machine - edges = [] for source in states: for symbol in range(8): @@ -404,18 +377,9 @@ def butterfly_process() -> EpsilonMachine: def ellison_fig9_forward() -> EpsilonMachine: """Forward ε-machine from Ellison et al., arXiv:1107.2168, Fig.~9.""" - from sofic.examples.processes import _edge_machine + from sofic.examples.processes import irreversible_two_state - return _edge_machine( - [ - ("A", "A", 0, 0.5), - ("A", "B", 1, 0.5), - ("B", "B", 1, 0.5), - ("B", "A", 2, 0.5), - ], - initial_distribution={"A": 0.5, "B": 0.5}, - normalize=False, - ) + return _relabel(irreversible_two_state(0.5, 0.5), symbols={"0": 0, "1": 1, "2": 2}) def tent_map_misiurewicz_a(symbolic: bool = False): @@ -878,7 +842,6 @@ def _stationary_distribution_from_joint_graph( graph: TransitionGraph, ) -> dict[tuple[str, str], Any]: from sofic.generators.prob import as_prob, has_symbolic, zeros - from sofic.generators.stationary import stationary_distribution_from_transition states = list(graph.states()) if not states: @@ -893,13 +856,7 @@ def _stationary_distribution_from_joint_graph( transition[source, target] = as_prob(transition[source, target]) + as_prob( transition_edge.data.get(ATTR_PROB, 0.0) ) - if symbolic: - pi = stationary_distribution_from_transition(transition) - return {states[position]: as_prob(pi[position]) for position in range(len(states))} - stationary = np.ones(len(states), dtype=float) / len(states) - for _ in range(20_000): - stationary = stationary @ np.asarray(transition, dtype=float) - return {states[position]: float(stationary[position]) for position in range(len(states))} + return _stationary_initial(states, transition) def _project_bidirectional_side( @@ -1004,7 +961,7 @@ def tent_map_misiurewicz_bidirectional_fig8(a: Any | None = None): else: try: reverse = EpsilonMachine.from_hmm(reverse_raw) - except Exception: + except SoficError: from sofic.generators.epsilon_machine import _row_normalized_presentation reverse = _row_normalized_presentation(reverse_raw) diff --git a/sofic/examples/processes.py b/sofic/examples/processes.py index 928ba77..6610917 100644 --- a/sofic/examples/processes.py +++ b/sofic/examples/processes.py @@ -15,12 +15,19 @@ import numpy as np from sofic.automata.transducers import MealyMachine +from sofic.examples._construction import _edge_machine, _relabel +from sofic.examples.epsilon_machines import ( + bernoulli, + even_process, + fair_coin, + golden_mean_forward, +) from sofic.generators.base import QuasiStochasticModel from sofic.generators.epsilon_machine import EpsilonMachine from sofic.generators.mealy import MealyHMM from sofic.graph import ATTR_EMISSION, ATTR_OUTPUT, ATTR_PROB, ATTR_QUASIPROB, ATTR_SYMBOL -RecurrentEpsilonMachine = EpsilonMachine +_STR_BITS = {0: "0", 1: "1"} def _require_machine_type(machine_type: Any, *allowed: type) -> None: @@ -39,79 +46,6 @@ def _as_alphabet(symbols: int | Sequence[Any]) -> tuple[Any, ...]: return tuple(symbols) -def _uniform_initial(states: Sequence[Hashable]) -> dict[Hashable, float]: - if not states: - return {} - mass = 1.0 / len(states) - return dict.fromkeys(states, mass) - - -def _stationary_initial( - states: Sequence[Hashable], - edges: Sequence[tuple[Hashable, Hashable, Any, float]], -) -> dict[Hashable, float]: - from sofic.generators.stationary import stationary_distribution_from_transition - - if not states: - return {} - index = {state: i for i, state in enumerate(states)} - transition = np.zeros((len(states), len(states)), dtype=float) - for source, target, _symbol, prob in edges: - transition[index[source], index[target]] += float(prob) - row_sums = transition.sum(axis=1) - if np.any(row_sums <= 0.0): - return _uniform_initial(states) - transition = transition / row_sums[:, None] - try: - pi = stationary_distribution_from_transition(transition) - except Exception: - return _uniform_initial(states) - return {state: float(pi[i]) for i, state in enumerate(states)} - - -def _normalize_edges( - edges: Sequence[tuple[Hashable, Hashable, Any, float]], -) -> list[tuple[Hashable, Hashable, Any, float]]: - row_totals: dict[Hashable, float] = {} - for source, _target, _symbol, prob in edges: - row_totals[source] = row_totals.get(source, 0.0) + float(prob) - normalized = [] - for source, target, symbol, prob in edges: - total = row_totals[source] - normalized.append((source, target, symbol, float(prob) / total if total else 0.0)) - return normalized - - -def _edge_machine( - edges: Iterable[tuple[Hashable, Hashable, Any, float]], - *, - machine_type: type[MealyHMM] = EpsilonMachine, - name: str | None = None, - initial_distribution: Mapping[Hashable, float] | None = None, - normalize: bool = True, - validate: bool = True, -) -> MealyHMM: - edge_list = list(edges) - if normalize: - edge_list = _normalize_edges(edge_list) - - states = list(dict.fromkeys([source for source, *_ in edge_list] + [target for _source, target, *_ in edge_list])) - symbols = frozenset(symbol for _source, _target, symbol, _prob in edge_list) - initial = dict(initial_distribution) if initial_distribution is not None else _stationary_initial(states, edge_list) - - machine = machine_type(initial_distribution=initial, observation_alphabet=symbols) - if name is not None: - machine.name = name - for state in states: - machine.graph.add_state(state) - for source, target, symbol, prob in edge_list: - if prob > 0.0: - machine.graph.add_transition(source, target, **{ATTR_EMISSION: symbol, ATTR_PROB: float(prob)}) - if validate: - machine.validate() - return machine - - def _from_string( spec: str, *, @@ -159,7 +93,7 @@ def _dirichlet(n: int, rng: Any = None) -> np.ndarray: def _compatible_machine_type(machine_type: Any, default: type[MealyHMM] = EpsilonMachine) -> type[MealyHMM]: if machine_type is None: return default - if machine_type in (EpsilonMachine, RecurrentEpsilonMachine, MealyHMM): + if machine_type in (EpsilonMachine, MealyHMM): return machine_type if isinstance(machine_type, str): lowered = machine_type.lower() @@ -170,13 +104,7 @@ def _compatible_machine_type(machine_type: Any, default: type[MealyHMM] = Epsilo return default -def ABC(p: float = 0.75, q: float = 0.25) -> EpsilonMachine: - if math.isclose(p, q): - return _from_string(f"A A 0 {p}; A A 1 {1 - p}", name="ABC Process") - return _from_string(f"A B 0 {p}; A B 1 {1 - p}; B A 0 {q}; B A 1 {1 - q}", name="ABC Process") - - -def AFC(n: int) -> EpsilonMachine: +def afc(n: int) -> EpsilonMachine: if n < 1: raise ValueError("n >= 1 required") spec = "B B 0; " @@ -186,7 +114,7 @@ def AFC(n: int) -> EpsilonMachine: return _from_string(spec, name=f"Almost Fair Coin Process, order {n}") -def AFC2(n: int) -> EpsilonMachine: +def afc2(n: int) -> EpsilonMachine: if n < 1: raise ValueError("n >= 1 required") spec = "A1 B1 1; B1 A1 0; " @@ -195,7 +123,7 @@ def AFC2(n: int) -> EpsilonMachine: return _from_string(spec, name=f"Almost Fair Coin 2 Process, order {n}") -def BandMerging(machine_type: Any = MealyHMM) -> MealyHMM: +def band_merging(machine_type: Any = MealyHMM) -> MealyHMM: cls = _compatible_machine_type(machine_type, MealyHMM) if cls is EpsilonMachine: edges = [ @@ -218,7 +146,7 @@ def BandMerging(machine_type: Any = MealyHMM) -> MealyHMM: ) -def BeadsOnNecklace( +def beads_on_necklace( machine_type: Any = MealyHMM, beads: Sequence[Any] | Sequence[Sequence[Any]] = ("a", "b"), necklace: Sequence[Any] = ("0", "1", "3"), @@ -263,7 +191,7 @@ def BeadsOnNecklace( return _edge_machine(edges, machine_type=MealyHMM, name="Beads On Necklace", normalize=False) -def BeforeAfter(machine_type: Any = MealyHMM, style: str = "simple") -> MealyHMM: +def before_after(machine_type: Any = MealyHMM, style: str = "simple") -> MealyHMM: _require_machine_type(machine_type, MealyHMM) if style == "simple": edges = [("A", "A", "0", 0.5), ("A", "B", "1", 0.5), ("B", "B", "0", 0.5), ("B", "A", "2", 0.5)] @@ -284,39 +212,26 @@ def BeforeAfter(machine_type: Any = MealyHMM, style: str = "simple") -> MealyHMM ] else: raise ValueError("style must be 'simple' or 'cayley'") - return _edge_machine(edges, machine_type=MealyHMM, name="BeforeAfter", normalize=False) + return _edge_machine(edges, machine_type=MealyHMM, name="before_after", normalize=False) -def BiasedCoin(bias: float, machine_type: Any = EpsilonMachine) -> EpsilonMachine: - cls = _compatible_machine_type(machine_type) - return _edge_machine( - [("A", "A", "1", bias), ("A", "A", "0", 1 - bias)], - machine_type=cls, - name=f"Coin, p = {bias}", - normalize=False, - ) - - -def FairCoin(machine_type: Any = EpsilonMachine) -> EpsilonMachine: - return BiasedCoin(0.5, machine_type=machine_type) - - -def RandomBiasedCoin(machine_type: Any = EpsilonMachine, rng: np.random.Generator | None = None) -> EpsilonMachine: +def random_biased_coin(machine_type: Any = EpsilonMachine, rng: np.random.Generator | None = None) -> EpsilonMachine: generator = rng if rng is not None else np.random.default_rng() - return BiasedCoin(float(generator.random()), machine_type=machine_type) + bias = float(generator.random()) + return _relabel(bernoulli(bias), machine_type=_compatible_machine_type(machine_type), name=f"Coin, p = {bias}") -def IID(k: int | Sequence[Any], machine_type: Any = EpsilonMachine) -> EpsilonMachine: +def iid(k: int | Sequence[Any], machine_type: Any = EpsilonMachine) -> EpsilonMachine: alphabet = _as_alphabet(k) return _edge_machine( [("A", "A", symbol, 1.0) for symbol in alphabet], machine_type=_compatible_machine_type(machine_type), - name=f"IID({len(alphabet)})", + name=f"iid({len(alphabet)})", normalize=True, ) -def BinaryMarkovChain( +def binary_markov_chain( p: float = 0.4, q: float = 0.3, a: float | None = None, @@ -324,24 +239,24 @@ def BinaryMarkovChain( machine_type: Any = EpsilonMachine, branch: Any = None, ) -> MealyHMM: - if machine_type in (EpsilonMachine, RecurrentEpsilonMachine, None): - return BMC_eM(p, q) + if machine_type in (EpsilonMachine, None): + return bmc_em(p, q) if isinstance(machine_type, str): lowered = machine_type.lower() if lowered == "generative": - return BMC_gen(p, q, branch=branch) + return bmc_gen(p, q, branch=branch) if lowered == "parametrized": if a is None or b is None: - raise ValueError("a and b are required for parametrized BinaryMarkovChain") - return BMC_param(p, q, a, b) + raise ValueError("a and b are required for parametrized binary_markov_chain") + return bmc_param(p, q, a, b) if lowered == "lohr": - return BMC_lohr(p) + return bmc_lohr(p) raise NotImplementedError -def BMC_eM(p: float, q: float) -> EpsilonMachine: +def bmc_em(p: float, q: float) -> EpsilonMachine: if math.isclose(p, 1 - q): - return BiasedCoin(bias=p) + return _relabel(bernoulli(p), name=f"Coin, p = {p}") return _edge_machine( [("A", "A", "0", 1 - p), ("A", "B", "1", p), ("B", "A", "0", q), ("B", "B", "1", 1 - q)], machine_type=EpsilonMachine, @@ -350,7 +265,7 @@ def BMC_eM(p: float, q: float) -> EpsilonMachine: ) -def BMC_gen(p: float, q: float, branch: Any = None) -> MealyHMM: +def bmc_gen(p: float, q: float, branch: Any = None) -> MealyHMM: del branch if p == 1 and q == 1: spec = "A B 1 1.; B A 0 1." @@ -371,26 +286,26 @@ def BMC_gen(p: float, q: float, branch: Any = None) -> MealyHMM: return _from_string(spec, machine_type=MealyHMM, name="BinaryMarkovChain gen", normalize=False) -def _BMC_param_get_a_range(p: float, q: float) -> list[float]: +def _bmc_param_get_a_range(p: float, q: float) -> list[float]: return [0, min(q, 1 - p)] -def _BMC_param_get_b_range(p: float, q: float) -> list[float]: +def _bmc_param_get_b_range(p: float, q: float) -> list[float]: return [max(q, 1 - p), 1] -def _BMC_param_check_a_range(p: float, q: float, a: float) -> bool: - low, high = _BMC_param_get_a_range(p, q) +def _bmc_param_check_a_range(p: float, q: float, a: float) -> bool: + low, high = _bmc_param_get_a_range(p, q) return low <= a <= high -def _BMC_param_check_b_range(p: float, q: float, b: float) -> bool: - low, high = _BMC_param_get_b_range(p, q) +def _bmc_param_check_b_range(p: float, q: float, b: float) -> bool: + low, high = _bmc_param_get_b_range(p, q) return low <= b <= high -def BMC_param(p: float, q: float, a: float, b: float) -> MealyHMM: - if not _BMC_param_check_a_range(p, q, a) or not _BMC_param_check_b_range(p, q, b): +def bmc_param(p: float, q: float, a: float, b: float) -> MealyHMM: + if not _bmc_param_check_a_range(p, q, a) or not _bmc_param_check_b_range(p, q, b): raise ValueError("a or b is outside the allowed range") probs = [ a * (b + p - 1) / (b - a), @@ -415,7 +330,7 @@ def BMC_param(p: float, q: float, a: float, b: float) -> MealyHMM: return _edge_machine(edges, machine_type=MealyHMM, name="BinaryMarkovChain parametrized", normalize=False) -def BMC_lohr(p: float) -> MealyHMM: +def bmc_lohr(p: float) -> MealyHMM: if p > 0.5: raise ValueError("currently requires p <= 0.5") return _edge_machine( @@ -435,7 +350,8 @@ def BMC_lohr(p: float) -> MealyHMM: ) -def Butterfly() -> EpsilonMachine: +def butterfly_two_branch() -> EpsilonMachine: + """cmpy's two-branch Butterfly (``h_mu = 1``); not :func:`~sofic.examples.butterfly_process` (8 symbols, ``h_mu = 3``).""" return _from_string( """ B C 0 .5; D E 0 .5; C A 1 .5; E A 1 .5; A B 2 .5; @@ -445,7 +361,7 @@ def Butterfly() -> EpsilonMachine: ) -def Cantor(machine_type: Any = MealyHMM) -> MealyHMM: +def cantor(machine_type: Any = MealyHMM) -> MealyHMM: _require_machine_type(machine_type, MealyHMM) edges = [ ("A", "A", "0", 0.55), @@ -458,7 +374,7 @@ def Cantor(machine_type: Any = MealyHMM) -> MealyHMM: return _edge_machine(edges, machine_type=MealyHMM, name="Cantor Process", normalize=False) -def CoupledGMPs( +def coupled_gmps( epsilon: float = 0.01, p: float = 0.5, alt: bool = True, machine_type: Any = EpsilonMachine ) -> EpsilonMachine: if epsilon < 0 or p < 0: @@ -503,7 +419,7 @@ def CoupledGMPs( ) -def UncoupledGMPs(p: float = 0.5, machine_type: Any = EpsilonMachine) -> EpsilonMachine: +def uncoupled_gmps(p: float = 0.5, machine_type: Any = EpsilonMachine) -> EpsilonMachine: if not 0 <= p <= 1: raise ValueError("p must be in [0, 1]") return _edge_machine( @@ -522,7 +438,7 @@ def UncoupledGMPs(p: float = 0.5, machine_type: Any = EpsilonMachine) -> Epsilon ) -def CyclicBranching(num_states: int, num_branchings: int, num_symbols: int = 2) -> MealyHMM: +def cyclic_branching(num_states: int, num_branchings: int, num_symbols: int = 2) -> MealyHMM: if num_branchings >= num_states: raise ValueError("number of branchings must be less than number of states") edges = [] @@ -542,8 +458,8 @@ def CyclicBranching(num_states: int, num_branchings: int, num_symbols: int = 2) ) -def Ehrenfest(p: float = 0.5, N: int = 5, machine_type: Any = EpsilonMachine) -> EpsilonMachine: - _require_machine_type(machine_type, EpsilonMachine, RecurrentEpsilonMachine) +def ehrenfest(p: float = 0.5, N: int = 5, machine_type: Any = EpsilonMachine) -> EpsilonMachine: + _require_machine_type(machine_type, EpsilonMachine) edges = [] for state in range(N + 1): edges.append((state, state, state, 1 - p)) @@ -551,43 +467,39 @@ def Ehrenfest(p: float = 0.5, N: int = 5, machine_type: Any = EpsilonMachine) -> edges.append((state, state + 1, state + 1, p * (N - state) / N)) for state in range(1, N + 1): edges.append((state, state - 1, state - 1, p * state / N)) - return _edge_machine(edges, machine_type=EpsilonMachine, name="Ehrenfest", normalize=False) + return _edge_machine(edges, machine_type=EpsilonMachine, name="ehrenfest", normalize=False) -def Even(machine_type: Any = EpsilonMachine, bias: float = 0.5) -> EpsilonMachine: - if machine_type in (EpsilonMachine, RecurrentEpsilonMachine, None): - edges = [("A", "A", "0", bias), ("A", "B", "1", 1 - bias), ("B", "A", "1", 1)] - return _edge_machine(edges, machine_type=EpsilonMachine, name="Even Process", normalize=False) - raise NotImplementedError - - -def RandomEven(machine_type: Any = EpsilonMachine, rng: np.random.Generator | None = None) -> EpsilonMachine: +def random_even(machine_type: Any = EpsilonMachine, rng: np.random.Generator | None = None) -> EpsilonMachine: return uniform_mealyhmm( - Even(), name="Random Even Process", create_using=_compatible_machine_type(machine_type), prng=rng + _relabel(even_process(), symbols=_STR_BITS), + name="Random Even Process", + create_using=_compatible_machine_type(machine_type), + prng=rng, ) -def EvenRedundant(machine_type: Any = EpsilonMachine, bias: float = 0.5) -> EpsilonMachine: - _require_machine_type(machine_type, EpsilonMachine, RecurrentEpsilonMachine) +def even_redundant(machine_type: Any = EpsilonMachine, bias: float = 0.5) -> EpsilonMachine: + _require_machine_type(machine_type, EpsilonMachine) return _from_string( f"A A 0 {bias}; A B 1 {1 - bias}; B C 1 1.; C C 0 {bias}; C D 1 {1 - bias}; D A 1 1.", name="Even Process (4-state)", ) -def ThreEven(machine_type: Any = EpsilonMachine) -> EpsilonMachine: - _require_machine_type(machine_type, EpsilonMachine, RecurrentEpsilonMachine) +def three_even(machine_type: Any = EpsilonMachine) -> EpsilonMachine: + _require_machine_type(machine_type, EpsilonMachine) return _from_string("A A 0 1; A B 1 1; A C 2 1; B A 1 1; C A 2 1;", name="ThreEven Process") -def Flower( +def flower( N: int = 4, M: int = 3, forward_dist: Sequence[float] | None = None, reverse_dists: np.ndarray | None = None, machine_type: Any = EpsilonMachine, ) -> EpsilonMachine: - _require_machine_type(machine_type, EpsilonMachine, RecurrentEpsilonMachine) + _require_machine_type(machine_type, EpsilonMachine) if N < 2 or M < 2: raise ValueError("N and M must be at least 2") if forward_dist is None: @@ -603,23 +515,23 @@ def Flower( return _edge_machine(edges, machine_type=EpsilonMachine, name="Flower Process", normalize=False) -def FourStateAlmostIID(delta: float = 0.2) -> EpsilonMachine: +def four_state_almost_iid(delta: float = 0.2) -> EpsilonMachine: delta = max(0.0, min(float(delta), 0.24)) p, q, r, s = 0.50 - 2 * delta, 0.50 - delta, 0.50 + delta, 0.50 + 2 * delta spec = f"A A 0 {p}; A B 1 {1 - p}; B C 0 {q}; B C 1 {1 - q}; C A 0 {r}; C D 1 {1 - r}; D A 0 {s}; D D 1 {1 - s};" return _from_string(spec, name="FourStateAlmostIID Process") -def Girvan_fig6b(machine_type: Any = EpsilonMachine, alpha: float = 0.5, pi: float = 0.4) -> EpsilonMachine: +def girvan_fig6b(machine_type: Any = EpsilonMachine, alpha: float = 0.5, pi: float = 0.4) -> EpsilonMachine: return _edge_machine( [("A", "A", "1", alpha), ("A", "P", "0", 1 - alpha), ("P", "A", "1", 1 - pi), ("P", "P", "0", pi)], machine_type=_compatible_machine_type(machine_type), - name="Girvan_fig6b", + name="girvan_fig6b", normalize=False, ) -def Girvan_fig6c( +def girvan_fig6c( machine_type: Any = EpsilonMachine, alpha: float = 0.5, pi: float = 0.4, rho: float = 0.3 ) -> EpsilonMachine: return _edge_machine( @@ -632,12 +544,12 @@ def Girvan_fig6c( ("R", "A", "1", 1 - rho), ], machine_type=_compatible_machine_type(machine_type), - name="Girvan_fig6c", + name="girvan_fig6c", normalize=False, ) -def Girvan_fig6d( +def girvan_fig6d( machine_type: Any = EpsilonMachine, alpha: float = 0.5, pi: float = 0.4, @@ -656,22 +568,18 @@ def Girvan_fig6d( ("I", "P", "0", 1 - iota), ], machine_type=_compatible_machine_type(machine_type), - name="Girvan_fig6d", + name="girvan_fig6d", normalize=False, ) -def GoldenMean(bias: float = 0.5, machine_type: Any = EpsilonMachine) -> EpsilonMachine: - _require_machine_type(machine_type, EpsilonMachine, RecurrentEpsilonMachine) - return _edge_machine( - [("A", "A", "1", 1 - bias), ("A", "B", "0", bias), ("B", "A", "1", 1)], - machine_type=EpsilonMachine, - name="Golden Mean Process", - normalize=False, - ) +def golden_mean_forbid_00(bias: float = 0.5, machine_type: Any = EpsilonMachine) -> EpsilonMachine: + """Golden mean forbidding ``00``; :func:`~sofic.examples.golden_mean` is its ``0 <-> 1`` mirror (forbids ``11``).""" + _require_machine_type(machine_type, EpsilonMachine) + return _relabel(golden_mean_forward(1 - bias), symbols=_STR_BITS, name="Golden Mean Process") -def RestrictedGM(k: int) -> EpsilonMachine: +def restricted_gm(k: int) -> EpsilonMachine: if k <= 0: raise ValueError("minimum k is 1") spec = "0 0 1 0.5; 0 1 0 0.5;" @@ -681,7 +589,7 @@ def RestrictedGM(k: int) -> EpsilonMachine: return _from_string(spec, name=f"Restricted Golden Mean Process, k={k}") -def StretchedGM(k: int) -> EpsilonMachine: +def stretched_gm(k: int) -> EpsilonMachine: if k <= 0: raise ValueError("minimum k is 1") spec = "0 0 1 0.5; 0 1 0 0.5;" @@ -691,7 +599,7 @@ def StretchedGM(k: int) -> EpsilonMachine: return _from_string(spec, name=f"Stretched Golden Mean Process, k={k}") -def RNGM(R: int, N: int, p: float = 0.5) -> EpsilonMachine: +def rn_gm(R: int, N: int, p: float = 0.5) -> EpsilonMachine: if R <= 0 or N <= 0 or R < N: raise ValueError("requires 1 <= N <= R") spec = f"0 0 1 {p}; 0 1 0 {1 - p};" @@ -703,7 +611,7 @@ def RNGM(R: int, N: int, p: float = 0.5) -> EpsilonMachine: return _from_string(spec, name=f"R-N Golden Mean Process, R={R} N={N}") -def RkGM(R: int, k: int, p: float = 0.5) -> EpsilonMachine: +def rk_gm(R: int, k: int, p: float = 0.5) -> EpsilonMachine: if R <= 0 or k <= 0: raise ValueError("R and k must be positive") spec = f"0 0 1 {p}; 0 1 0 {1 - p};" @@ -715,13 +623,16 @@ def RkGM(R: int, k: int, p: float = 0.5) -> EpsilonMachine: return _from_string(spec, name=f"R-k Golden Mean Process, R={R} k={k}") -def RandomGoldenMean(machine_type: Any = EpsilonMachine, rng: np.random.Generator | None = None) -> EpsilonMachine: +def random_golden_mean(machine_type: Any = EpsilonMachine, rng: np.random.Generator | None = None) -> EpsilonMachine: return uniform_mealyhmm( - GoldenMean(), name="Random Golden Mean Process", create_using=_compatible_machine_type(machine_type), prng=rng + golden_mean_forbid_00(), + name="Random Golden Mean Process", + create_using=_compatible_machine_type(machine_type), + prng=rng, ) -def GoldenMeanGHMM() -> QuasiStochasticModel: +def golden_mean_ghmm() -> QuasiStochasticModel: q = QuasiStochasticModel(initial_quasidistribution={"A": 2 / 3, "B": 1 / 3}) q.observation_alphabet = frozenset({"0", "1"}) q.name = "Golden Mean Process" @@ -739,7 +650,7 @@ def GoldenMeanGHMM() -> QuasiStochasticModel: return q -def NonunifilarGoldenMean(bias: float = 0.5, free: float = 2 / 3) -> MealyHMM: +def nonunifilar_golden_mean(bias: float = 0.5, free: float = 2 / 3) -> MealyHMM: pGM = bias pA = 1 / (1 + pGM) pB = pGM / (1 + pGM) @@ -767,7 +678,7 @@ def NonunifilarGoldenMean(bias: float = 0.5, free: float = 2 / 3) -> MealyHMM: return _edge_machine(edges, machine_type=MealyHMM, name=f"Nonunifilar Golden Mean, p = {bias:.2f}", normalize=False) -def IrreversibleTwoState(p: float = 0.5, q: float = 0.5, machine_type: Any = EpsilonMachine) -> EpsilonMachine: +def irreversible_two_state(p: float = 0.5, q: float = 0.5, machine_type: Any = EpsilonMachine) -> EpsilonMachine: return _edge_machine( [("A", "A", "0", p), ("A", "B", "1", 1 - p), ("B", "B", "1", q), ("B", "A", "2", 1 - q)], machine_type=_compatible_machine_type(machine_type), @@ -776,8 +687,8 @@ def IrreversibleTwoState(p: float = 0.5, q: float = 0.5, machine_type: Any = Eps ) -def Ising(machine_type: Any = EpsilonMachine, J: float = 1.0, B: float = 0.3, T: float = 1.0) -> EpsilonMachine: - _require_machine_type(machine_type, EpsilonMachine, RecurrentEpsilonMachine) +def ising(machine_type: Any = EpsilonMachine, J: float = 1.0, B: float = 0.3, T: float = 1.0) -> EpsilonMachine: + _require_machine_type(machine_type, EpsilonMachine) beta = 1.0 / T rad = (np.sinh(beta * B) ** 2 + np.exp(-4 * beta * J)) ** 0.5 two_p = 1.0 - (2 * np.exp(-4 * beta * J)) / (rad * (np.cosh(beta * B) + rad)) @@ -794,10 +705,10 @@ def Ising(machine_type: Any = EpsilonMachine, J: float = 1.0, B: float = 0.3, T: ) -def Lollipop( +def lollipop( N: int, M: int, p: float = 0.5, q: float = 0.5, r: float = 0.1, machine_type: Any = EpsilonMachine ) -> EpsilonMachine: - _require_machine_type(machine_type, EpsilonMachine, RecurrentEpsilonMachine) + _require_machine_type(machine_type, EpsilonMachine) hns = [str(ind) for ind in range(N)] sns = [str(ind) for ind in range(N, N + 2 * (M - 1) + 1)] edges = [] @@ -814,18 +725,18 @@ def Lollipop( edges.append((sns[2 * (M - 1) - 1], sns[last], "1", 1 - r)) edges.append((sns[2 * (M - 1) - 1], sns[last], "0", r)) edges.append((sns[last], hns[0], "2", 1)) - return _edge_machine(edges, machine_type=EpsilonMachine, name="Lollipop", normalize=False) + return _edge_machine(edges, machine_type=EpsilonMachine, name="lollipop", normalize=False) -def LogicMachine( +def logic_machine( logic: str, bias: float | Sequence[float] = 0.5, noise: float | Sequence[float] = 0.5, minimize: bool = True ) -> MealyHMM: del minimize - if logic == "RRX": - return RRX(machine_type=MealyHMM) - if logic == "Rn1C": - return Rn1C(noise=float(np.atleast_1d(noise)[0]), bias=float(np.atleast_1d(bias)[0])) - raise NotImplementedError("LogicMachine currently supports the common RRX and Rn1C constructors") + if logic == "rrx": + return rrx(machine_type=MealyHMM) + if logic == "rn1c": + return rn1c(noise=float(np.atleast_1d(noise)[0]), bias=float(np.atleast_1d(bias)[0])) + raise NotImplementedError("logic_machine currently supports the common 'RRX' and 'Rn1C' logics") def markov_skeleton(R: int, k: int | Sequence[Any], join: bool | None = None) -> MealyHMM: @@ -849,7 +760,7 @@ def markov_skeleton(R: int, k: int | Sequence[Any], join: bool | None = None) -> return _edge_machine(edges, machine_type=MealyHMM, name=f"Order-{R} Markov skeleton") -def Misiurewicz(machine_type: Any = MealyHMM) -> MealyHMM: +def misiurewicz(machine_type: Any = MealyHMM) -> MealyHMM: _require_machine_type(machine_type, MealyHMM) return _edge_machine( [ @@ -867,7 +778,7 @@ def Misiurewicz(machine_type: Any = MealyHMM) -> MealyHMM: ) -def MisiurewiczSimplified(machine_type: Any = MealyHMM) -> MealyHMM: +def misiurewicz_simplified(machine_type: Any = MealyHMM) -> MealyHMM: _require_machine_type(machine_type, MealyHMM) return _edge_machine( [ @@ -885,7 +796,7 @@ def MisiurewiczSimplified(machine_type: Any = MealyHMM) -> MealyHMM: ) -def MisiurewiczUniform(machine_type: Any = MealyHMM) -> MealyHMM: +def misiurewicz_uniform(machine_type: Any = MealyHMM) -> MealyHMM: _require_machine_type(machine_type, MealyHMM) return _edge_machine( [ @@ -903,32 +814,24 @@ def MisiurewiczUniform(machine_type: Any = MealyHMM) -> MealyHMM: ) -def Multiple3(machine_type: Any = MealyHMM, bias: float = 0.5) -> MealyHMM: - return MultipleN(3, machine_type=machine_type, bias=bias) +def multiple3(machine_type: Any = MealyHMM, bias: float = 0.5) -> MealyHMM: + return multiple_n(3, machine_type=machine_type, bias=bias) -def Multiple4(machine_type: Any = MealyHMM, bias: float = 0.5) -> MealyHMM: - return MultipleN(4, machine_type=machine_type, bias=bias) +def multiple4(machine_type: Any = MealyHMM, bias: float = 0.5) -> MealyHMM: + return multiple_n(4, machine_type=machine_type, bias=bias) -def MultipleN(n: int, machine_type: Any = MealyHMM, bias: float = 0.5) -> MealyHMM: - _require_machine_type(machine_type, MealyHMM, EpsilonMachine, RecurrentEpsilonMachine) +def multiple_n(n: int, machine_type: Any = MealyHMM, bias: float = 0.5) -> MealyHMM: + _require_machine_type(machine_type, MealyHMM, EpsilonMachine) edges = [(0, 0, "0", bias), (0, 1, "1", 1 - bias)] for x in range(1, int(n)): edges.append((x, 0 if x == n - 1 else x + 1, "1", 1)) - cls = EpsilonMachine if machine_type in (EpsilonMachine, RecurrentEpsilonMachine) else MealyHMM + cls = EpsilonMachine if machine_type is EpsilonMachine else MealyHMM return _edge_machine(edges, machine_type=cls, name=f"'Multiples of {n}' Process", normalize=False) -def Nemo(machine_type: Any = EpsilonMachine, p: float = 0.5, q: float = 0.5) -> EpsilonMachine: - _require_machine_type(machine_type, EpsilonMachine, RecurrentEpsilonMachine) - return _from_string( - f"A A 1 {p}; A B 0 {1 - p}; B C 0 1; C A 0 {1 - q}; C A 1 {q};", - name="Nemo Process", - ) - - -def NemoRedundant(machine_type: Any = MealyHMM, p: float = 0.5, q: float = 0.5) -> MealyHMM: +def nemo_redundant(machine_type: Any = MealyHMM, p: float = 0.5, q: float = 0.5) -> MealyHMM: _require_machine_type(machine_type, MealyHMM) return _from_string( f""" @@ -940,7 +843,7 @@ def NemoRedundant(machine_type: Any = MealyHMM, p: float = 0.5, q: float = 0.5) ) -def NoisyPeriod2(noise: float = 0.5) -> EpsilonMachine: +def noisy_period2(noise: float = 0.5) -> EpsilonMachine: return _edge_machine( [("A", "B", "0", 1), ("B", "A", "0", noise), ("B", "A", "1", 1 - noise)], machine_type=EpsilonMachine, @@ -949,14 +852,8 @@ def NoisyPeriod2(noise: float = 0.5) -> EpsilonMachine: ) -def NRPS() -> EpsilonMachine: - from sofic.examples.epsilon_machines import noisy_random_phase_slip - - return noisy_random_phase_slip() - - -def Odd(machine_type: Any = EpsilonMachine, bias1: float = 0.5, bias2: float = 0.5) -> EpsilonMachine: - _require_machine_type(machine_type, EpsilonMachine, RecurrentEpsilonMachine) +def odd(machine_type: Any = EpsilonMachine, bias1: float = 0.5, bias2: float = 0.5) -> EpsilonMachine: + _require_machine_type(machine_type, EpsilonMachine) edges = [ ("A", "A", "0", bias1), ("A", "B", "1", 1 - bias1), @@ -967,7 +864,7 @@ def Odd(machine_type: Any = EpsilonMachine, bias1: float = 0.5, bias2: float = 0 return _edge_machine(edges, machine_type=EpsilonMachine, name="Odd Process", normalize=False) -def OddGHMM(variant: int = 1) -> QuasiStochasticModel: +def odd_ghmm(variant: int = 1) -> QuasiStochasticModel: matrices = ( {"0": [[0.5, 0, 0], [1.0, 0, 0], [1.0, 0, 0]], "1": [[0.5, -0.5, 0.5], [0, 0, 0], [0.5, 0, -0.5]]} if variant == 1 @@ -990,23 +887,23 @@ def OddGHMM(variant: int = 1) -> QuasiStochasticModel: return q -def EvenOdd(machine_type: Any = EpsilonMachine) -> EpsilonMachine: - _require_machine_type(machine_type, EpsilonMachine, RecurrentEpsilonMachine) +def even_odd(machine_type: Any = EpsilonMachine) -> EpsilonMachine: + _require_machine_type(machine_type, EpsilonMachine) return _from_string("A B 0 0.5; B A 0 1; A C 1 0.5; C B 0 0.5; C D 1 0.5; D C 1 1;", name="EvenOdd Process") -def ThreEvenOdd(machine_type: Any = EpsilonMachine) -> EpsilonMachine: - _require_machine_type(machine_type, EpsilonMachine, RecurrentEpsilonMachine) +def three_even_odd(machine_type: Any = EpsilonMachine) -> EpsilonMachine: + _require_machine_type(machine_type, EpsilonMachine) return _from_string( "A A 0 1; A B 1 1; B A 1 1; A C 2 1; C A 0 1; C B 1 1; C D 2 1; D C 2 1;", name="ThreEvenOdd Process" ) -def Period(P: int) -> MealyHMM: - return Periodic("0" * (P - 1) + "1") +def period(P: int) -> MealyHMM: + return periodic("0" * (P - 1) + "1") -def Periodic(word: Sequence[Any], reduce: bool = True) -> MealyHMM: +def periodic(word: Sequence[Any], reduce: bool = True) -> MealyHMM: base = _word_period(word) if reduce else word edges = [] for i, symbol in enumerate(base): @@ -1016,52 +913,52 @@ def Periodic(word: Sequence[Any], reduce: bool = True) -> MealyHMM: ) -def Period1(machine_type: Any = MealyHMM) -> MealyHMM: - return Periodic("1") if machine_type is MealyHMM else _from_string("A A 1 1", name="Period-1 Process") +def period1(machine_type: Any = MealyHMM) -> MealyHMM: + return periodic("1") if machine_type is MealyHMM else _from_string("A A 1 1", name="Period-1 Process") -def Period2(machine_type: Any = MealyHMM) -> MealyHMM: - return Periodic("01") if machine_type is MealyHMM else _from_string("A B 0 1; B A 1 1", name="Period-2 Process") +def period2(machine_type: Any = MealyHMM) -> MealyHMM: + return periodic("01") if machine_type is MealyHMM else _from_string("A B 0 1; B A 1 1", name="Period-2 Process") -def Period4(machine_type: Any = MealyHMM) -> MealyHMM: +def period4(machine_type: Any = MealyHMM) -> MealyHMM: return ( - Periodic("1110") + periodic("1110") if machine_type is MealyHMM else _from_string("A B 1 1; B C 1 1; C D 1 1; D A 0 1", name="Period-4 Process") ) -def Period7(machine_type: Any = MealyHMM) -> MealyHMM: +def period7(machine_type: Any = MealyHMM) -> MealyHMM: if machine_type is not MealyHMM: raise NotImplementedError - return Periodic("10101110") + return periodic("10101110") -def Period8(machine_type: Any = MealyHMM) -> MealyHMM: +def period8(machine_type: Any = MealyHMM) -> MealyHMM: if machine_type is not MealyHMM: raise NotImplementedError - return Periodic("10101110") + return periodic("10101110") -def Period12(machine_type: Any = MealyHMM) -> MealyHMM: +def period12(machine_type: Any = MealyHMM) -> MealyHMM: if machine_type is not MealyHMM: raise NotImplementedError - return Periodic("101011101110") + return periodic("101011101110") -def Period16(machine_type: Any = MealyHMM) -> MealyHMM: +def period16(machine_type: Any = MealyHMM) -> MealyHMM: if machine_type is not MealyHMM: raise NotImplementedError - return Periodic("1010111011101110") + return periodic("1010111011101110") -def PerturbedCoin(p: float = 0.2, q: float | None = None, machine_type: Any = EpsilonMachine) -> MealyHMM: +def perturbed_coin(p: float = 0.2, q: float | None = None, machine_type: Any = EpsilonMachine) -> MealyHMM: if p == 0.5: - return FairCoin() + return fair_coin() if q is None: q = p - if machine_type in (EpsilonMachine, RecurrentEpsilonMachine, None): + if machine_type in (EpsilonMachine, None): return _edge_machine( [("A", "A", "0", 1 - p), ("A", "B", "1", p), ("B", "A", "0", q), ("B", "B", "1", 1 - q)], machine_type=EpsilonMachine, @@ -1072,15 +969,16 @@ def PerturbedCoin(p: float = 0.2, q: float | None = None, machine_type: Any = Ep if lowered in {"lohr", "generative"}: if q != p: raise ValueError("Lohr/generative variants only support q == p") - return BMC_lohr(p) if lowered == "lohr" else BMC_gen(p, p) + return bmc_lohr(p) if lowered == "lohr" else bmc_gen(p, p) raise NotImplementedError(f"cannot build machine type {machine_type!r}") -def PSB() -> EpsilonMachine: +def phase_slip_backtrack_cmpy() -> EpsilonMachine: + """cmpy's Phase-Slip Backtrack; a different process from :func:`~sofic.examples.epsilon_machines.phase_slip_backtrack`.""" return _from_string("A B 1; A D 0; B B 1; B C 0; C D 0; D A 1", name="Phase-Slip Backtrack") -def RIP(p: float = 0.5, q: float = 0.5, reverse: bool = False) -> EpsilonMachine: +def rip(p: float = 0.5, q: float = 0.5, reverse: bool = False) -> EpsilonMachine: if reverse: spec = f"D E 1 {1 - p * q}; D F 0 {p * q}; E D 1 {(1 - p) / (1 - p * q)}; E G 0 {p * (1 - q) / (1 - p * q)}; F G 0 1; G D 1 1" else: @@ -1088,7 +986,7 @@ def RIP(p: float = 0.5, q: float = 0.5, reverse: bool = False) -> EpsilonMachine return _from_string(spec, name="Random Insertion Process") -def Rn1C(noise: float = 0.5, bias: float = 0.5) -> EpsilonMachine: +def rn1c(noise: float = 0.5, bias: float = 0.5) -> EpsilonMachine: return _edge_machine( [ ("A", "B", "0", bias), @@ -1103,8 +1001,8 @@ def Rn1C(noise: float = 0.5, bias: float = 0.5) -> EpsilonMachine: ) -def Rn1N(machine_type: Any = EpsilonMachine, bias: float = 0.5, noise: float = 0.1) -> EpsilonMachine: - _require_machine_type(machine_type, EpsilonMachine, RecurrentEpsilonMachine) +def rn1n(machine_type: Any = EpsilonMachine, bias: float = 0.5, noise: float = 0.1) -> EpsilonMachine: + _require_machine_type(machine_type, EpsilonMachine) return _edge_machine( [ ("A", "B", "0", bias), @@ -1119,8 +1017,8 @@ def Rn1N(machine_type: Any = EpsilonMachine, bias: float = 0.5, noise: float = 0 ) -def RRX(machine_type: Any = EpsilonMachine) -> MealyHMM: - if machine_type in (EpsilonMachine, RecurrentEpsilonMachine, None): +def rrx(machine_type: Any = EpsilonMachine) -> MealyHMM: + if machine_type in (EpsilonMachine, None): edges = [ ("S", "0", "0", 0.5), ("S", "1", "1", 0.5), @@ -1131,7 +1029,7 @@ def RRX(machine_type: Any = EpsilonMachine) -> MealyHMM: ("00|11", "S", "0", 1), ("01|10", "S", "1", 1), ] - return _edge_machine(edges, machine_type=EpsilonMachine, name="RRX", normalize=False) + return _edge_machine(edges, machine_type=EpsilonMachine, name="rrx", normalize=False) if machine_type is MealyHMM: return _edge_machine( [ @@ -1153,19 +1051,19 @@ def RRX(machine_type: Any = EpsilonMachine) -> MealyHMM: raise NotImplementedError -def RRXRO(machine_type: Any = EpsilonMachine) -> EpsilonMachine: - _require_machine_type(machine_type, EpsilonMachine, RecurrentEpsilonMachine) +def rrxro(machine_type: Any = EpsilonMachine) -> EpsilonMachine: + _require_machine_type(machine_type, EpsilonMachine) return _from_string( """ S 0 0 .5; S 1 1 .5; 0 00 0 .5; 0 01 1 .5; 1 10 0 .5; 1 11 1 .5; 01 orTrue 1 1; 10 orTrue 1 1; 00 000 0 1; 000 orFalse 0 .5; 000 orTrue 1 .5; 11 110 0 1; 110 orFalse 0 .5; 110 orTrue 1 .5; orFalse S 0 1; orTrue S 1 1; """, - name="RRXRO", + name="rrxro", ) -def SNS(machine_type: Any = MealyHMM) -> MealyHMM: +def sns(machine_type: Any = MealyHMM) -> MealyHMM: _require_machine_type(machine_type, MealyHMM) return _edge_machine( [("A", "A", "1", 0.5), ("A", "B", "1", 0.5), ("B", "A", "0", 0.5), ("B", "B", "1", 0.5)], @@ -1175,10 +1073,10 @@ def SNS(machine_type: Any = MealyHMM) -> MealyHMM: ) -def ThreeHundred( +def three_hundred( machine_type: Any = EpsilonMachine, biases: tuple[float, float, float] = (0.5, 0.5, 0.5) ) -> EpsilonMachine: - _require_machine_type(machine_type, EpsilonMachine, RecurrentEpsilonMachine) + _require_machine_type(machine_type, EpsilonMachine) p, q, r = biases spec = f"A B 0 {p}; A C 1 {1 - p}; B D 0 1; C E 0 1; D F 0 {q}; D F 1 {1 - q}; E A 0 1; F A 0 {r}; F A 1 {1 - r};" return _from_string(spec, name="ThreeHundred Process") @@ -1275,24 +1173,24 @@ def _transducer( return machine -def GMtoEven(bias: float = 0.5, create_using: Any = None) -> MealyMachine: +def gm_to_even(bias: float = 0.5, create_using: Any = None) -> MealyMachine: del bias, create_using return _transducer([("A", "A", "1", "0", 1), ("A", "B", "0", "1", 1), ("B", "A", "1", "1", 1)], name="GM to Even") -def RCT(bias: float = 0.5, create_using: Any = None) -> MealyMachine: +def rct(bias: float = 0.5, create_using: Any = None) -> MealyMachine: del create_using return _transducer( [("A", "B", "0", "0", bias), ("A", "C", "0", "1", bias), ("B", "A", "0", "0", 1), ("C", "A", "1", "1", 1)] ) -def BitFlip(create_using: Any = None) -> MealyMachine: +def bit_flip(create_using: Any = None) -> MealyMachine: del create_using return _transducer([("A", "A", "0", "1", 1), ("A", "A", "1", "0", 1)]) -def FlipEveryOther(parity: str = "Even", create_using: Any = None) -> MealyMachine: +def flip_every_other(parity: str = "Even", create_using: Any = None) -> MealyMachine: del create_using initial = "A" if parity == "Even" else "B" return _transducer( @@ -1301,7 +1199,7 @@ def FlipEveryOther(parity: str = "Even", create_using: Any = None) -> MealyMachi ) -def Delay(length: int = 2, symbols: int | Sequence[Any] = 2, create_using: Any = None) -> MealyMachine: +def delay(length: int = 2, symbols: int | Sequence[Any] = 2, create_using: Any = None) -> MealyMachine: del create_using alphabet = tuple(map(str, _as_alphabet(symbols))) edges = [] @@ -1313,7 +1211,7 @@ def Delay(length: int = 2, symbols: int | Sequence[Any] = 2, create_using: Any = return _transducer(edges) -def TwoPerm(symbols: int | Sequence[Any] = 2, create_using: Any = None) -> MealyMachine: +def two_perm(symbols: int | Sequence[Any] = 2, create_using: Any = None) -> MealyMachine: del create_using alphabet = tuple(map(str, _as_alphabet(symbols))) edges = [] @@ -1326,28 +1224,28 @@ def TwoPerm(symbols: int | Sequence[Any] = 2, create_using: Any = None) -> Mealy return _transducer(edges, initial="??") -def BinaryChannel(p: float = 0.0, q: float = 0.0, create_using: Any = None) -> MealyMachine: +def binary_channel(p: float = 0.0, q: float = 0.0, create_using: Any = None) -> MealyMachine: del create_using return _transducer( [("A", "A", "0", "0", 1 - p), ("A", "A", "0", "1", p), ("A", "A", "1", "1", 1 - q), ("A", "A", "1", "0", q)] ) -def SlidingNOR(create_using: Any = None) -> MealyMachine: +def sliding_nor(create_using: Any = None) -> MealyMachine: del create_using return _transducer( [("A", "A", "0", "1", 1), ("A", "B", "1", "0", 1), ("B", "A", "0", "0", 1), ("B", "B", "1", "0", 1)] ) -def Parity(create_using: Any = None) -> MealyMachine: +def parity(create_using: Any = None) -> MealyMachine: del create_using return _transducer( [("A", "A", "0", "0", 1), ("A", "B", "1", "1", 1), ("B", "A", "0", "0", 1), ("B", "A", "1", "0", 1)] ) -def GME(create_using: Any = None) -> MealyMachine: +def gme(create_using: Any = None) -> MealyMachine: del create_using return _transducer( [ @@ -1361,93 +1259,87 @@ def GME(create_using: Any = None) -> MealyMachine: ) -def BinaryChannelET(p: float = 0.1, q: float = 0.2) -> Any: +def binary_channel_et(p: float = 0.1, q: float = 0.2) -> Any: """Memoryless binary channel as its minimal (single-state) ε-transducer.""" from sofic.generators.epsilon_transducer import EpsilonTransducer - return EpsilonTransducer.from_channel(BinaryChannel(p, q)) + return EpsilonTransducer.from_channel(binary_channel(p, q)) -def GMtoEvenET() -> Any: +def gm_to_even_et() -> Any: """Golden-mean-to-even map as an ε-transducer (Barnett & Crutchfield 2015).""" from sofic.generators.epsilon_transducer import EpsilonTransducer - return EpsilonTransducer.from_channel(GMtoEven()) + return EpsilonTransducer.from_channel(gm_to_even()) processes = [ - "ABC", - "BandMerging", - "BeadsOnNecklace", - "BeforeAfter", - "BiasedCoin", - "BinaryMarkovChain", - "Butterfly", - "Cantor", - "CoupledGMPs", - "GoldenMean", - "IrreversibleTwoState", - "NonunifilarGoldenMean", - "Ehrenfest", - "Even", - "EvenOdd", - "EvenRedundant", - "FairCoin", - "Flower", - "FourStateAlmostIID", - "Girvan_fig6b", - "Girvan_fig6c", - "Girvan_fig6d", - "Ising", - "Lollipop", - "Misiurewicz", - "MisiurewiczSimplified", - "MisiurewiczUniform", - "Multiple3", - "Multiple4", - "Nemo", - "NemoRedundant", - "NoisyPeriod2", - "NRPS", - "Odd", - "Period1", - "Period2", - "Period4", - "Period7", - "Period8", - "Period12", - "Period16", - "PerturbedCoin", - "PSB", - "RandomBiasedCoin", - "RandomGoldenMean", - "RandomEven", - "RIP", - "Rn1C", - "Rn1N", - "RRX", - "RRXRO", - "SNS", - "ThreeHundred", - "ThreEven", - "ThreEvenOdd", + "band_merging", + "beads_on_necklace", + "before_after", + "binary_markov_chain", + "butterfly_two_branch", + "cantor", + "coupled_gmps", + "golden_mean_forbid_00", + "irreversible_two_state", + "nonunifilar_golden_mean", + "ehrenfest", + "even_odd", + "even_redundant", + "flower", + "four_state_almost_iid", + "girvan_fig6b", + "girvan_fig6c", + "girvan_fig6d", + "ising", + "lollipop", + "misiurewicz", + "misiurewicz_simplified", + "misiurewicz_uniform", + "multiple3", + "multiple4", + "nemo_redundant", + "noisy_period2", + "odd", + "period1", + "period2", + "period4", + "period7", + "period8", + "period12", + "period16", + "perturbed_coin", + "phase_slip_backtrack_cmpy", + "random_biased_coin", + "random_golden_mean", + "random_even", + "rip", + "rn1c", + "rn1n", + "rrx", + "rrxro", + "sns", + "three_hundred", + "three_even", + "three_even_odd", ] -nonergodic_generators = ["UncoupledGMPs"] +nonergodic_generators = ["uncoupled_gmps"] transducers = [ - "GMtoEven", - "RCT", - "BitFlip", - "FlipEveryOther", - "Delay", - "TwoPerm", - "BinaryChannel", - "SlidingNOR", - "Parity", - "GME", + "gm_to_even", + "rct", + "bit_flip", + "flip_every_other", + "delay", + "two_perm", + "binary_channel", + "sliding_nor", + "parity", + "gme", ] epsilon_transducers = [ - "BinaryChannelET", - "GMtoEvenET", + "binary_channel_et", + "gm_to_even_et", ] process_list = [globals()[name] for name in processes] @@ -1457,25 +1349,25 @@ def GMtoEvenET() -> Any: __all__ = processes + nonergodic_generators + transducers + epsilon_transducers __all__ += [ - "AFC", - "AFC2", - "BMC_eM", - "BMC_gen", - "BMC_param", - "BMC_lohr", - "CyclicBranching", - "GoldenMeanGHMM", - "IID", - "LogicMachine", + "afc", + "afc2", + "bmc_em", + "bmc_gen", + "bmc_param", + "bmc_lohr", + "cyclic_branching", + "golden_mean_ghmm", + "iid", + "logic_machine", "markov_skeleton", - "MultipleN", - "OddGHMM", - "Period", - "Periodic", - "RestrictedGM", - "StretchedGM", - "RkGM", - "RNGM", + "multiple_n", + "odd_ghmm", + "period", + "periodic", + "restricted_gm", + "stretched_gm", + "rk_gm", + "rn_gm", "uniform_mealyhmm", "uniform_mealymc", "processes", @@ -1484,8 +1376,8 @@ def GMtoEvenET() -> Any: "transducer_list", "epsilon_transducers", "epsilon_transducer_list", - "_BMC_param_get_a_range", - "_BMC_param_get_b_range", - "_BMC_param_check_a_range", - "_BMC_param_check_b_range", + "_bmc_param_get_a_range", + "_bmc_param_get_b_range", + "_bmc_param_check_a_range", + "_bmc_param_check_b_range", ] diff --git a/sofic/examples/tetris.py b/sofic/examples/tetris.py index f83381e..c80a9d1 100644 --- a/sofic/examples/tetris.py +++ b/sofic/examples/tetris.py @@ -22,7 +22,7 @@ import numpy as np -from sofic.examples.processes import _edge_machine +from sofic.examples._construction import _edge_machine from sofic.generators.epsilon_machine import EpsilonMachine TETROMINOES: tuple[str, ...] = ("I", "J", "L", "O", "S", "T", "Z") diff --git a/sofic/exceptions.py b/sofic/exceptions.py index b8a7658..e9d39ff 100644 --- a/sofic/exceptions.py +++ b/sofic/exceptions.py @@ -43,3 +43,7 @@ class LumpabilityError(SoficValidationError): class InfiniteTransductionError(SoficError): """Raised when a finite input has infinitely many transducer outputs.""" + + +class NonWellMatchedLanguageError(SoficValidationError): + """Raised when a construction defined for well-matched languages meets a pending call or return.""" diff --git a/sofic/generators/__init__.py b/sofic/generators/__init__.py index d0866e3..92ce391 100644 --- a/sofic/generators/__init__.py +++ b/sofic/generators/__init__.py @@ -16,7 +16,6 @@ synergistic_information_flow, transfer_entropy, ) -from sofic.generators.epsilon_inference import cssr, spectral, subtree_merge, suggest_lmax from sofic.generators.epsilon_machine import EpsilonMachine from sofic.generators.epsilon_transducer import EpsilonTransducer from sofic.generators.lumping import LumpabilityError, is_lumpable, lump, normalize_partition @@ -38,12 +37,6 @@ from sofic.generators.pfa import ProbabilisticFiniteAutomaton from sofic.generators.quasi_realization import QuasiRealization from sofic.generators.stack_hmm import HiddenMarkovStackModel -from sofic.generators.stack_inference import ( - fit_stack_hmm_mle, - learn_stack_hmm_papni, - stack_cssr, - stack_subtree_merge, -) from sofic.generators.topological_epsilon_enumeration import ( count_topological_epsilon_machines, epsilon_machine_to_idfa_string, @@ -76,17 +69,9 @@ "LumpabilityError", "channel_statistical_complexity", "driven_entropy_rate", - "cssr", "is_lumpable", "lump", "normalize_partition", - "spectral", - "subtree_merge", - "suggest_lmax", - "fit_stack_hmm_mle", - "learn_stack_hmm_papni", - "stack_cssr", - "stack_subtree_merge", "directed_information", "independent_pair_generator", "intrinsic_information_flow", diff --git a/sofic/generators/alternative_complexity.py b/sofic/generators/alternative_complexity.py index c65d118..68e5ad0 100644 --- a/sofic/generators/alternative_complexity.py +++ b/sofic/generators/alternative_complexity.py @@ -100,5 +100,5 @@ def _stationary_vector(transition: np.ndarray) -> np.ndarray | None: try: return stationary_distribution_from_transition(transition) - except Exception: + except (ValueError, np.linalg.LinAlgError): return None diff --git a/sofic/generators/base.py b/sofic/generators/base.py index 276ce6b..bc28e17 100644 --- a/sofic/generators/base.py +++ b/sofic/generators/base.py @@ -79,19 +79,18 @@ def reverse(self) -> Self: """ from sofic.generators.reversal import is_markov_like, time_reverse_stochastic - if not is_markov_like(self): - from sofic.generators.epsilon_machine import EpsilonMachine - from sofic.generators.mealy import MealyHMM - from sofic.generators.moore import MooreHMM + if is_markov_like(self): + return time_reverse_stochastic(self) - if isinstance(self, (MealyHMM, MooreHMM)): - from sofic.generators.epsilon_machine import EpsilonMachine + from sofic.generators.epsilon_machine import EpsilonMachine + from sofic.generators.mealy import MealyHMM + from sofic.generators.moore import MooreHMM - if isinstance(self, EpsilonMachine): - return EpsilonMachine.from_time_reversed(self) - return EpsilonMachine.from_hmm(time_reverse_stochastic(self)) - raise NotImplementedError("time-reversed generators with edge emissions require EpsilonMachine.from_hmm") - return time_reverse_stochastic(self) + if isinstance(self, EpsilonMachine): + return EpsilonMachine.from_time_reversed(self) + if isinstance(self, (MealyHMM, MooreHMM)): + return EpsilonMachine.from_hmm(time_reverse_stochastic(self)) + raise NotImplementedError("time-reversed generators with edge emissions require EpsilonMachine.from_hmm") class HiddenMarkovModel(StochasticModel): @@ -104,39 +103,39 @@ def __init__(self, observation_alphabet: frozenset[Any] | None = None, **kwargs: self.observation_alphabet = observation_alphabet if observation_alphabet is not None else frozenset() def sample(self, n: int, rng: np.random.Generator | None = None) -> tuple[list[Any], list[Hashable]]: - from sofic.generators.hmm_inference import sample + from sofic.generators.sampling import sample return sample(self, n, rng) def log_likelihood(self, observations: Sequence[Any]) -> float: - from sofic.generators.hmm_inference import log_likelihood + from sofic.inference.hmm import log_likelihood return log_likelihood(self, observations) - def forward(self, observations: Sequence[Any], *, scaled: bool = False) -> np.ndarray: - from sofic.generators.hmm_inference import forward + def forward(self, observations: Sequence[Any], *, normalize: bool = False) -> np.ndarray: + from sofic.inference.hmm import forward - return forward(self, observations, scaled=scaled) + return forward(self, observations, normalize=normalize) - def backward(self, observations: Sequence[Any], *, scaled: bool = False) -> np.ndarray: - from sofic.generators.hmm_inference import backward + def backward(self, observations: Sequence[Any], *, normalize: bool = False) -> np.ndarray: + from sofic.inference.hmm import backward - return backward(self, observations, scaled=scaled) + return backward(self, observations, normalize=normalize) def viterbi(self, observations: Sequence[Any]) -> list[Hashable]: - from sofic.generators.hmm_inference import viterbi + from sofic.inference.hmm import viterbi return viterbi(self, observations) def smooth(self, observations: Sequence[Any]) -> np.ndarray: """Return fixed-interval smoothed marginals ``gamma[t, s]``.""" - from sofic.generators.hmm_inference import smooth + from sofic.inference.hmm import smooth return smooth(self, observations) def two_slice_marginals(self, observations: Sequence[Any]) -> np.ndarray: """Return two-slice smoothed marginals ``xi[t, i, j]``.""" - from sofic.generators.hmm_inference import two_slice_marginals + from sofic.inference.hmm import two_slice_marginals return two_slice_marginals(self, observations) @@ -152,9 +151,9 @@ def baum_welch( ) -> tuple[MealyHMM, list[float]]: """Fit parameters by Baum-Welch EM, returning ``(fitted_model, loglik_trace)``. - ``n_restarts`` and ``rng`` are as in :func:`~sofic.generators.hmm_inference.baum_welch`. + ``n_restarts`` and ``rng`` are as in :func:`~sofic.inference.hmm.baum_welch`. """ - from sofic.generators.hmm_inference import baum_welch + from sofic.inference.hmm import baum_welch return baum_welch( self, @@ -168,19 +167,19 @@ def baum_welch( def score(self, observations: Sequence[Any]) -> dict[tuple[Hashable, Any, Hashable], float]: """Return the log-likelihood gradient (Fisher identity) over edge parameters.""" - from sofic.generators.hmm_inference import score + from sofic.inference.hmm import score return score(self, observations) def observed_information(self, observations: Sequence[Any]) -> np.ndarray: """Return the observed information matrix (Louis' identity).""" - from sofic.generators.hmm_inference import observed_information + from sofic.inference.hmm import observed_information return observed_information(self, observations) def standard_errors(self, observations: Sequence[Any]) -> dict[tuple[Hashable, Any, Hashable], float]: """Return asymptotic standard errors of the free edge parameters.""" - from sofic.generators.hmm_inference import standard_errors + from sofic.inference.hmm import standard_errors return standard_errors(self, observations) @@ -193,16 +192,16 @@ def entropy_rate(self) -> float: return entropy_rate_hmm(self) - def joint_block_distribution(self, history_length: int = 1) -> Any: + def joint_block_distribution(self, block_length: int = 2) -> Any: from sofic.generators.measures import joint_block_distribution - return joint_block_distribution(self, history_length=history_length) + return joint_block_distribution(self, block_length=block_length) def words_of_length(self, length: int) -> dict[tuple[Any, ...], float]: """Return observed words of ``length`` and their probabilities.""" - from sofic.generators.words import hmm_words_of_length + from sofic.generators.words import _hmm_words_of_length - return hmm_words_of_length(self, length) + return _hmm_words_of_length(self, length) def word_probability( self, @@ -211,9 +210,9 @@ def word_probability( start: Hashable | Mapping[Hashable, float] | Sequence[float] | np.ndarray | None = None, ) -> float: """Return the probability of an observed finite word.""" - from sofic.generators.words import hmm_word_probability + from sofic.generators.words import _hmm_word_probability - return hmm_word_probability(self, word, start=start) + return _hmm_word_probability(self, word, start=start) def log_word_probability( self, @@ -222,9 +221,9 @@ def log_word_probability( start: Hashable | Mapping[Hashable, float] | Sequence[float] | np.ndarray | None = None, ) -> float: """Return ``log2`` of an observed finite-word probability.""" - from sofic.generators.words import hmm_log_word_probability + from sofic.generators.words import _hmm_log_word_probability - return hmm_log_word_probability(self, word, start=start) + return _hmm_log_word_probability(self, word, start=start) def word_probabilities( self, @@ -234,9 +233,9 @@ def word_probabilities( sparse: bool = True, ) -> dict[tuple[Any, ...], float]: """Return observed-word probabilities for one or more lengths.""" - from sofic.generators.words import hmm_word_probabilities + from sofic.generators.words import _hmm_word_probabilities - return hmm_word_probabilities(self, lengths, start=start, sparse=sparse) + return _hmm_word_probabilities(self, lengths, start=start, sparse=sparse) def conditional_word_probability( self, @@ -246,9 +245,9 @@ def conditional_word_probability( start: Hashable | Mapping[Hashable, float] | Sequence[float] | np.ndarray | None = None, ) -> float: """Return ``P(word | condition)``.""" - from sofic.generators.words import hmm_conditional_word_probability + from sofic.generators.words import _hmm_conditional_word_probability - return hmm_conditional_word_probability(self, word, condition, start=start) + return _hmm_conditional_word_probability(self, word, condition, start=start) def is_equal_process( self, @@ -312,16 +311,16 @@ def stationary_quasidistribution(self) -> np.ndarray: return stationary_quasidistribution(self) - def transition_matrices(self) -> dict[Any, np.ndarray]: - from sofic.generators.quasi_inference import transition_matrices + def symbol_matrices(self) -> dict[Any, np.ndarray]: + from sofic.generators.quasi_inference import symbol_matrices - return transition_matrices(self) + return symbol_matrices(self) def words_of_length(self, length: int) -> dict[tuple[Any, ...], float]: """Return words of ``length`` and their signed quasiprobabilities.""" - from sofic.generators.words import quasi_words_of_length + from sofic.generators.words import _quasi_words_of_length - return quasi_words_of_length(self, length) + return _quasi_words_of_length(self, length) def collision_entropy(self) -> float: from sofic.generators.measures import collision_entropy diff --git a/sofic/generators/block_convergence.py b/sofic/generators/block_convergence.py index 7800981..a3b592b 100644 --- a/sofic/generators/block_convergence.py +++ b/sofic/generators/block_convergence.py @@ -8,6 +8,7 @@ import numpy as np +from sofic.exceptions import SoficError from sofic.generators._word_measures import ( _block_caekl, _block_coinformation, @@ -203,7 +204,7 @@ def _anatomy_curves( def _exact_anatomy_scalars(machine: EpsilonMachine) -> dict[str, float] | None: try: bidir = machine.to_bidirectional() - except Exception: + except (SoficError, ValueError, NotImplementedError, np.linalg.LinAlgError): return None h_mu = float(bidir.entropy_rate()) rho_mu = float(bidir.predicted_information()) diff --git a/sofic/generators/block_entropy.py b/sofic/generators/block_entropy.py index 64a8ca0..a6288f6 100644 --- a/sofic/generators/block_entropy.py +++ b/sofic/generators/block_entropy.py @@ -9,6 +9,8 @@ import numpy as np +from sofic.exceptions import SoficError + if TYPE_CHECKING: from sofic.generators.epsilon_machine import EpsilonMachine @@ -327,7 +329,10 @@ def block_entropy_estimates( entropy_asymptote = excess_entropy + h_mu_l crypticity_estimate = state_block_entropy - block_state_entropy - crypticity = float(crypticity_estimate[-1]) if crypticity_estimate.size else 0.0 + if use_exact: + crypticity = statistical_complexity - excess_entropy + else: + crypticity = float(crypticity_estimate[-1]) if crypticity_estimate.size else 0.0 cm = _cm_extension_curves( lengths, @@ -423,10 +428,10 @@ def _block_entropy_curves( def _stationary_symbol_matrices(machine: EpsilonMachine) -> tuple[np.ndarray, dict[Any, np.ndarray]]: - from sofic.generators.hmm_inference import _emission_transition_tensors + from sofic.generators.matrices import symbol_matrices pi = machine.stationary_distribution() - _, raw_matrices = _emission_transition_tensors(machine) + raw_matrices = symbol_matrices(machine.to_mealy()) n = len(pi) zero = np.zeros((n, n), dtype=float) matrices = { @@ -453,7 +458,7 @@ def _excess_entropy( ) -> float: try: return float(machine.to_bidirectional().excess_entropy()) - except Exception: + except (SoficError, ValueError, NotImplementedError, np.linalg.LinAlgError): return _block_entropy_excess_entropy(machine, entropy_rate=entropy_rate, block_entropy=block_entropy) @@ -498,7 +503,7 @@ def _estimated_excess_entropy(machine: EpsilonMachine, estimate: np.ndarray, *, if use_exact: try: return float(machine.excess_entropy()) - except Exception: + except (SoficError, ValueError, NotImplementedError, np.linalg.LinAlgError): pass return float(estimate[-1]) if estimate.size else 0.0 diff --git a/sofic/generators/channel_measures.py b/sofic/generators/channel_measures.py index 0ff3c58..0c12e45 100644 --- a/sofic/generators/channel_measures.py +++ b/sofic/generators/channel_measures.py @@ -4,7 +4,7 @@ structural quantities are defined relative to a driving input process. Each measure here drives the transducer with a supplied input generator, forms the joint ``(input, output)`` process via -:func:`~sofic.automata.transducer_operations.compose_tg`, and reads off the +:func:`~sofic.automata.transducer_operations.compose_transducer_generator`, and reads off the quantity -- reusing the directional-flow estimators in :mod:`sofic.generators.directional_flow`. """ @@ -24,9 +24,9 @@ def driven_joint_generator(transducer: MealyMachine, input_process: HiddenMarkovModel) -> MealyHMM: """Return the joint ``(input, output)`` generator induced by ``input_process``.""" - from sofic.automata.transducer_operations import compose_tg + from sofic.automata.transducer_operations import compose_transducer_generator - return compose_tg(transducer, input_process, joint=True) + return compose_transducer_generator(transducer, input_process, joint=True) def channel_statistical_complexity(transducer: MealyMachine, input_process: HiddenMarkovModel) -> float: @@ -35,6 +35,8 @@ def channel_statistical_complexity(transducer: MealyMachine, input_process: Hidd ``S`` is the transducer's causal-state component of the driven joint process, weighted by its stationary occupation distribution. """ + from sofic.generators.stochastic import shannon_entropy + joint = driven_joint_generator(transducer, input_process) idx = joint.reindex() if len(idx) == 0: @@ -44,11 +46,7 @@ def channel_statistical_complexity(transducer: MealyMachine, input_process: Hidd for state, weight in zip(idx.states, pi, strict=True): transducer_state = state[1] if isinstance(state, tuple) and len(state) == 2 else state mass[transducer_state] += float(weight) - probs = np.array([value for value in mass.values() if value > 0.0], dtype=float) - if probs.size == 0: - return 0.0 - probs = probs / probs.sum() - return float(-(probs * np.log2(probs)).sum()) + return shannon_entropy(mass.values(), normalize=True) def driven_entropy_rate(transducer: MealyMachine, input_process: HiddenMarkovModel) -> float: diff --git a/sofic/generators/directional_flow.py b/sofic/generators/directional_flow.py index e5e9f6c..6aaf8ba 100644 --- a/sofic/generators/directional_flow.py +++ b/sofic/generators/directional_flow.py @@ -19,36 +19,22 @@ def _require_dit(): def _pair_block_distribution(generator: HiddenMarkovModel, *, history: int) -> Any: """Joint law over flattened ``(x0, y0, x1, y1, ...)`` windows.""" - from sofic.generators.hmm_inference import _stationary_emission_tensors + from sofic.generators.matrices import emission_tensors + from sofic.generators.words import _enumerate_words, _matrix_step dit = _require_dit() # Directional-flow statistics describe the stationary joint process, so weight # the initial state by the stationary law rather than ``initial_distribution``. - pi, joint = _stationary_emission_tensors(generator) + pi, joint = emission_tensors(generator, policy="stationary") + if any(not isinstance(symbol, tuple) or len(symbol) != 2 for symbol in joint): + raise TypeError("generator must emit length-2 tuple symbols") - block_length = history + 1 ones = np.ones(len(pi), dtype=float) outcomes: list[tuple[Any, ...]] = [] probs: list[float] = [] - - def walk( - mass: np.ndarray, - prefix: tuple[Any, ...], - steps_remaining: int, - ) -> None: - if steps_remaining == 0: - outcomes.append(prefix) - probs.append(float(mass @ ones)) - return - for symbol, matrix in joint.items(): - if not isinstance(symbol, tuple) or len(symbol) != 2: - raise TypeError("generator must emit length-2 tuple symbols") - next_mass = mass @ matrix - if next_mass.sum() <= 0.0: - continue - walk(next_mass, prefix + (symbol[0], symbol[1]), steps_remaining - 1) - - walk(pi.copy(), (), block_length) + for word, mass in _enumerate_words(list(joint), history + 1, pi.copy(), _matrix_step(joint, len(pi))): + outcomes.append(tuple(value for pair in word for value in pair)) + probs.append(float(mass @ ones)) total = sum(probs) if total > 0.0: probs = [p / total for p in probs] diff --git a/sofic/generators/epsilon_inference.py b/sofic/generators/epsilon_inference.py deleted file mode 100644 index 488be70..0000000 --- a/sofic/generators/epsilon_inference.py +++ /dev/null @@ -1,1081 +0,0 @@ -"""Sample-based ε-machine reconstruction (CSSR, subtree merging, and spectral). - -CSSR follows Shalizi, Shalizi & Crutchfield (arXiv:cs/0210025). Subtree merging -follows Crutchfield & Young (PRL 1989; PRE 1994). Spectral reconstruction learns -a weighted finite automaton by Hankel SVD :cite:`Balle2014,Hsu2012` and extracts -causal states as mixed states of the learned operators :cite:`Ellison2009`. -""" - -from __future__ import annotations - -import zlib -from collections import Counter, defaultdict -from collections.abc import Callable, Iterable, Sequence -from dataclasses import dataclass, field -from functools import lru_cache -from typing import Any, ClassVar, Literal - -import numpy as np -from scipy import stats - -from sofic.exceptions import StochasticValidationError -from sofic.generators.epsilon_machine import EpsilonMachine -from sofic.graph import ATTR_EMISSION, ATTR_PROB, TransitionGraph - -History = tuple[Any, ...] - -#: Morph-equality tests: G-test, chi-squared, total-variation threshold, or Monte Carlo exact G-test. -MorphTest = Literal["g", "chi2", "tv", "exact"] - - -@dataclass -class SuffixCounts: - """Empirical counts of histories and following symbols in a sequence.""" - - alphabet: tuple[Any, ...] - history_counts: Counter[History] = field(default_factory=Counter) - next_counts: dict[History, Counter[Any]] = field(default_factory=lambda: defaultdict(Counter)) - - #: History key used as the fallback for an empty history set (overridden by stack counts). - empty_history: ClassVar[History] = () - - @classmethod - def from_sequence( - cls, - sequence: Sequence[Any], - *, - alphabet: Sequence[Any] | None = None, - max_length: int | None = None, - ) -> SuffixCounts: - seq = tuple(sequence) - if not seq: - raise ValueError("sequence must be non-empty") - alphabet = tuple(sorted(set(seq), key=repr)) if alphabet is None else tuple(alphabet) - unknown = set(seq) - set(alphabet) - if unknown: - raise ValueError(f"symbols {unknown!r} not in alphabet") - max_len = max_length if max_length is not None else len(seq) - counts = cls(alphabet=alphabet) - n = len(seq) - for t in range(n): - for length in range(0, min(t, max_len) + 1): - history = seq[t - length : t] - counts.history_counts[history] += 1 - nxt = seq[t] - counts.next_counts[history][nxt] += 1 - return counts - - def morph(self, history: History, *, smoothing: float = 0.0) -> dict[Any, float]: - """MLE (optional additive smoothing) of P(next symbol | history).""" - counts = self.next_counts.get(history, Counter()) - total = sum(counts.values()) - if total == 0: - uniform = 1.0 / len(self.alphabet) - return dict.fromkeys(self.alphabet, uniform) - denom = total + smoothing * len(self.alphabet) - return {symbol: (counts.get(symbol, 0) + smoothing) / denom for symbol in self.alphabet} - - def state_morph(self, histories: set[History], *, smoothing: float = 0.0) -> dict[Any, float]: - """Weighted average of history morphs with weights from occurrence counts.""" - weights = {history: float(self.history_counts.get(history, 0)) for history in histories} - total_weight = sum(weights.values()) - if total_weight <= 0.0: - return self.morph(self.empty_history, smoothing=smoothing) - result = dict.fromkeys(self.alphabet, 0.0) - for history, weight in weights.items(): - morph = self.morph(history, smoothing=smoothing) - for symbol in self.alphabet: - result[symbol] += weight * morph[symbol] - return {symbol: prob / total_weight for symbol, prob in result.items()} - - def marginal_morph(self) -> dict[Any, float]: - """Global next-symbol distribution (IID morph at L=0).""" - counts = Counter() - for _history, counter in self.next_counts.items(): - counts.update(counter) - grand = sum(counts.values()) - if grand == 0: - uniform = 1.0 / len(self.alphabet) - return dict.fromkeys(self.alphabet, uniform) - return {symbol: counts.get(symbol, 0) / grand for symbol in self.alphabet} - - def restricted_to(self, histories: set[History]) -> SuffixCounts: - """Return a plain :class:`SuffixCounts` proxy limited to ``histories``. - - The morph/comparison helpers only read the history sets handed to them, so - stack inference can reuse them by projecting its configuration counts onto a - flat proxy without changing any results. - """ - proxy = SuffixCounts(alphabet=self.alphabet) - proxy.history_counts = Counter({h: self.history_counts.get(h, 0) for h in histories}) - proxy.next_counts = defaultdict(Counter) - for history in histories: - proxy.next_counts[history] = self.next_counts.get(history, Counter()) - return proxy - - -def _observed_counts_for_morph( - counts: SuffixCounts, - histories: set[History], -) -> Counter[Any]: - observed = Counter() - for history in histories: - observed.update(counts.next_counts.get(history, Counter())) - return observed - - -def _contingency_rows( - counts: SuffixCounts, - left_histories: set[History], - right_histories: set[History], -) -> np.ndarray | None: - left_obs = _observed_counts_for_morph(counts, left_histories) - right_obs = _observed_counts_for_morph(counts, right_histories) - active = [symbol for symbol in counts.alphabet if left_obs.get(symbol, 0) + right_obs.get(symbol, 0) > 0] - if not active: - return None - table = np.array( - [ - [left_obs.get(symbol, 0) for symbol in active], - [right_obs.get(symbol, 0) for symbol in active], - ], - dtype=float, - ) - if np.allclose(table[0], table[1]): - return None - if table.shape[1] < 2: - left_total = table[0].sum() - right_total = table[1].sum() - if left_total == 0.0 or right_total == 0.0: - return None - left_prob = table[0, 0] / left_total - right_prob = table[1, 0] / right_total - if np.isclose(left_prob, right_prob): - return None - return table - - -def _g_statistic(table: np.ndarray) -> float | None: - """G-test statistic of a contingency table, as ``scipy.stats.chi2_contingency`` computes it. - - Includes Yates' correction for one degree of freedom, and returns ``None`` if an - expected count is zero. Inlined because the test runs once per pair of - histories, and the general scipy routine dominated inference time. - """ - expected = table.sum(axis=1, keepdims=True) * table.sum(axis=0, keepdims=True) / table.sum() - if np.any(expected == 0): - return None - observed = table - if (table.shape[0] - 1) * (table.shape[1] - 1) == 1: - diff = expected - observed - observed = observed + np.sign(diff) * np.minimum(0.5, np.abs(diff)) - with np.errstate(divide="ignore", invalid="ignore"): - terms = np.where(observed > 0, observed * np.log(observed / expected), 0.0) - return 2.0 * float(terms.sum()) - - -@lru_cache(maxsize=256) -def _chi2_critical(alpha: float, dof: int) -> float: - return float(stats.chi2.ppf(1.0 - alpha, dof)) - - -#: Monte Carlo tables drawn per ``"exact"`` morph test. -_EXACT_DRAWS = 999 - -#: Smallest expected count at which the ``"exact"`` test trusts the chi-squared limit. -_EXACT_MIN_EXPECTED = 5.0 - - -def _exact_g_pvalue(table: np.ndarray) -> float: - """Monte Carlo p-value of the G statistic among tables with the same margins. - - Tables are drawn uniformly given both margins (``scipy.stats.random_table``), - the exact null of equal morphs. The generator is seeded from the table itself, - so reconstruction stays deterministic. - """ - counts = table.astype(np.int64) - rows, cols = counts.sum(axis=1), counts.sum(axis=0) - expected = np.outer(rows, cols) / counts.sum() - - def g(observed: np.ndarray) -> np.ndarray: - with np.errstate(divide="ignore", invalid="ignore"): - terms = np.where(observed > 0, observed * np.log(observed / expected), 0.0) - return 2.0 * terms.sum(axis=(-2, -1)) - - rng = np.random.default_rng(zlib.crc32(counts.tobytes())) - draws = stats.random_table(rows, cols, seed=rng).rvs(size=_EXACT_DRAWS) - observed = g(counts.astype(float)) - extreme = np.sum(g(draws.astype(float)) >= observed - 1e-9 * max(1.0, observed)) - return float((1 + extreme) / (1 + _EXACT_DRAWS)) - - -def _exact_significant(table: np.ndarray, alpha: float) -> bool: - """The ``"exact"`` decision for a contingency table of next-symbol counts. - - Uses the Monte Carlo exact G-test when an expected count is below - ``_EXACT_MIN_EXPECTED`` and the asymptotic G-test otherwise. - """ - if np.any(table.sum(axis=1) == 0): - return False - expected = np.outer(table.sum(axis=1), table.sum(axis=0)) / table.sum() - if expected.min() < _EXACT_MIN_EXPECTED: - return _exact_g_pvalue(table) < alpha - statistic = _g_statistic(table) - if statistic is None or not np.isfinite(statistic): - return False - return statistic > _chi2_critical(alpha, max(1, table.shape[1] - 1)) - - -def _bonferroni_alpha( - counts: SuffixCounts, - alpha: float, - *, - max_length: int, - min_count: int, - suffix_length: Callable[[History], int] = len, -) -> float: - """``alpha`` divided by the number of suffixes eligible for a split test.""" - eligible = sum( - 1 - for history, following in counts.next_counts.items() - if 0 < suffix_length(history) <= max_length and sum(following.values()) >= max(1, min_count) - ) - return alpha / max(1, eligible) - - -def morphs_differ( - counts: SuffixCounts, - left_histories: set[History], - right_histories: set[History], - *, - alpha: float = 0.05, - test: MorphTest = "g", - delta: float = 0.0, -) -> bool: - """Return whether two history sets have significantly different morphs. - - ``"g"`` and ``"chi2"`` use the chi-squared limit, which is unreliable when - expected counts are small. ``"exact"`` instead compares the G statistic with - tables drawn uniformly given the observed margins whenever an expected count is - below 5, and uses the G-test otherwise. - """ - if test == "tv": - left = counts.state_morph(left_histories) - right = counts.state_morph(right_histories) - distance = 0.5 * sum(abs(left[s] - right[s]) for s in counts.alphabet) - return distance > delta - - table = _contingency_rows(counts, left_histories, right_histories) - if table is None: - return False - if test == "exact": - return _exact_significant(table, alpha) - if test == "g": - statistic = _g_statistic(table) - if statistic is None or not np.isfinite(statistic): - return False - return statistic > _chi2_critical(alpha, max(1, table.shape[1] - 1)) - try: - statistic, p_value, _dof, expected = stats.chi2_contingency(table) - except ValueError: - return False - if np.any(expected == 0): - return False - return float(p_value) < alpha - - -def morph_test_score( - counts: SuffixCounts, - left_histories: set[History], - right_histories: set[History], - *, - test: MorphTest = "g", -) -> float: - """Score for matching morphs (lower is more similar).""" - if test == "tv": - left = counts.state_morph(left_histories) - right = counts.state_morph(right_histories) - return 0.5 * sum(abs(left[s] - right[s]) for s in counts.alphabet) - table = _contingency_rows(counts, left_histories, right_histories) - if table is None: - return 0.0 - try: - if test in ("g", "exact"): - statistic = _g_statistic(table) - return statistic if statistic is not None and np.isfinite(statistic) else 0.0 - statistic, _p, _dof, _expected = stats.chi2_contingency(table) - return float(statistic) - except ValueError: - return 0.0 - - -def _default_lmax(n: int, alphabet_size: int, min_count: int) -> int: - if alphabet_size <= 0: - return 1 - return max(1, min(15, n // max(1, alphabet_size * min_count))) - - -def _grow_history(history: History, symbol: Any) -> History: - """Default successor: append the symbol without truncation (flat ε-machine CSSR).""" - return history + (symbol,) - - -def _cssr_homogenize( - counts: SuffixCounts, - *, - Lmax: int, - alpha: float, - test: MorphTest, - successor_fn: Callable[[History, Any], History] = _grow_history, -) -> tuple[dict[int, set[History]], dict[History, int]]: - """Return state id -> histories and history -> state id.""" - states: dict[int, set[History]] = {0: {counts.empty_history}} - history_to_state: dict[History, int] = {counts.empty_history: 0} - next_state_id = 1 - - for _length in range(Lmax + 1): - for state_id in sorted(states): - histories = set(states[state_id]) - for history in list(histories): - for symbol in counts.alphabet: - child = successor_fn(history, symbol) - if child in history_to_state: - continue - if counts.history_counts.get(child, 0) == 0: - continue - child_histories = {child} - if morphs_differ( - counts, - histories, - child_histories, - alpha=alpha, - test=test, - ): - best_state: int | None = None - best_score = float("inf") - for candidate_id, candidate_histories in states.items(): - if morphs_differ( - counts, - candidate_histories, - child_histories, - alpha=alpha, - test=test, - ): - continue - score = morph_test_score( - counts, - candidate_histories, - child_histories, - test=test, - ) - if score < best_score: - best_score = score - best_state = candidate_id - if best_state is None: - best_state = next_state_id - states[next_state_id] = set() - next_state_id += 1 - states[best_state].add(child) - history_to_state[child] = best_state - else: - states[state_id].add(child) - history_to_state[child] = state_id - return states, history_to_state - - -def _cssr_determinize( - states: dict[int, set[History]], - history_to_state: dict[History, int], - counts: SuffixCounts, - *, - length: int, - successor_fn: Callable[[History, Any], History] = _grow_history, -) -> dict[int, set[History]]: - """Split homogeneous states until transitions are unifilar.""" - current = {state_id: set(histories) for state_id, histories in states.items()} - changed = True - next_state_id = max(current) + 1 if current else 0 - - while changed: - changed = False - for state_id in sorted(current): - histories = current[state_id] - if len(histories) <= 1: - continue - for symbol in counts.alphabet: - buckets: dict[int, set[History]] = defaultdict(set) - for history in histories: - if counts.next_counts.get(history, Counter()).get(symbol, 0) == 0: - continue - child = successor_fn(history, symbol) - target = history_to_state.get(child) - if target is None: - continue - buckets[target].add(history) - if len(buckets) <= 1: - continue - # Keep the largest bucket in the original state; split others. - ordered = sorted(buckets.items(), key=lambda item: (-len(item[1]), min(item[1]))) - keep_target, keep_histories = ordered[0] - current[state_id] = keep_histories - for _target, split_histories in ordered[1:]: - new_id = next_state_id - next_state_id += 1 - current[new_id] = split_histories - for history in split_histories: - history_to_state[history] = new_id - changed = True - break - if changed: - break - return current - - -def _merge_similar_states( - states: dict[int, set[History]], - history_to_state: dict[History, int], - counts: SuffixCounts, - *, - alpha: float, - test: MorphTest, -) -> dict[int, set[History]]: - """Merge inferred states whose pooled morphs are statistically indistinguishable. - - Merging on the morph alone can fuse states with incompatible ``symbol -> - successor`` maps, yielding a non-unifilar partition. Callers must re-run - :func:`_cssr_determinize` afterwards to restore unifilarity. - """ - current = {state_id: set(histories) for state_id, histories in states.items()} - changed = True - while changed: - changed = False - state_ids = sorted(current) - for index, left_id in enumerate(state_ids): - if left_id not in current: - continue - for right_id in state_ids[index + 1 :]: - if right_id not in current: - continue - if morphs_differ( - counts, - current[left_id], - current[right_id], - alpha=alpha, - test=test, - ): - continue - current[left_id].update(current.pop(right_id)) - for history in current[left_id]: - history_to_state[history] = left_id - changed = True - break - if changed: - break - return current - - -def _drop_transient_states( - states: dict[int, set[History]], - history_to_state: dict[History, int], - counts: SuffixCounts, - *, - length: int, - successor_fn: Callable[[History, Any], History] = _grow_history, -) -> dict[int, set[History]]: - """Keep only states in bottom strongly connected components.""" - import networkx as nx - - successors: dict[int, dict[Any, set[int]]] = defaultdict(lambda: defaultdict(set)) - for state_id, histories in states.items(): - for history in histories: - for symbol in counts.alphabet: - if counts.next_counts.get(history, Counter()).get(symbol, 0) == 0: - continue - target = history_to_state.get(successor_fn(history, symbol)) - if target is None: - continue - successors[state_id][symbol].add(target) - - graph = nx.DiGraph() - for state_id in states: - graph.add_node(state_id) - for state_id, by_symbol in successors.items(): - for targets in by_symbol.values(): - for target in targets: - graph.add_edge(state_id, target) - - if graph.number_of_edges() == 0: - return states - - recurrent: set[int] = set() - for component in nx.strongly_connected_components(graph): - if not component: - continue - subgraph = graph.subgraph(component) - has_cycle = subgraph.number_of_edges() > 0 and ( - len(component) > 1 or any(subgraph.has_edge(node, node) for node in component) - ) - if not has_cycle: - continue - outgoing = any(graph.has_edge(v, w) for v in component for w in graph.nodes if w not in component) - if not outgoing: - recurrent.update(component) - - if not recurrent: - return states - return {state_id: histories for state_id, histories in states.items() if state_id in recurrent} - - -def _cssr_default_lmax(n: int, alphabet_size: int) -> int: - """A third of ``log_k n``, between 1 and 10. - - Each length-``L`` word is then seen about ``n ** (2/3)`` times. Longer suffixes - multiply the number of significance tests, and with them the false splits. - """ - k = max(2, alphabet_size) - return max(1, min(10, int(np.log(n) / (3 * np.log(k))))) - - -def _suffix_homogenize( - counts: SuffixCounts, - *, - Lmax: int, - alpha: float, - test: MorphTest, - min_count: int = 1, -) -> list[set[History]]: - """CSSR homogenization: grow suffixes one symbol into the past, up to length ``Lmax``. - - Each child suffix ``a x`` stays in its parent's state unless its next-symbol - distribution differs significantly; then it joins the most similar state that - does not differ, or starts a new one. As in :cite:`Shalizi2002`, states keep - the suffixes of every length they collect. Suffixes seen fewer than ``min_count`` - times are not tested: the significance test is unreliable on so few counts. - """ - states: list[set[History]] = [{()}] - for length in range(Lmax): - for parent_id in range(len(states)): - parent = states[parent_id] - for history in sorted((h for h in parent if len(h) == length), key=repr): - for symbol in counts.alphabet: - child = (symbol, *history) - if sum(counts.next_counts.get(child, Counter()).values()) < max(1, min_count): - continue - if not morphs_differ(counts, parent, {child}, alpha=alpha, test=test): - parent.add(child) - continue - best_id, best_score = None, float("inf") - for candidate_id, candidate in enumerate(states): - if candidate_id == parent_id: - continue - if morphs_differ(counts, candidate, {child}, alpha=alpha, test=test): - continue - score = morph_test_score(counts, candidate, {child}, test=test) - if score < best_score: - best_id, best_score = candidate_id, score - if best_id is None: - states.append({child}) - else: - states[best_id].add(child) - return states - - -def _suffix_successor(history: History, symbol: Any, Lmax: int) -> History: - """The suffix that follows ``history`` on ``symbol``: extended, or truncated at ``Lmax``.""" - extended = (*history, symbol) - return extended[1:] if len(extended) > Lmax else extended - - -def _suffix_edges( - states: list[set[History]], - counts: SuffixCounts, - alive: set[int], - *, - Lmax: int, - alpha: float, - test: MorphTest, - resolve: bool = True, -) -> dict[int, dict[Any, dict[int, set[History]]]]: - """Successor states of each alive state, by symbol, with the suffixes that lead there. - - A suffix shorter than ``Lmax`` moves to the state holding its one-symbol extension. - A length-``Lmax`` suffix must drop its oldest symbol, which can forget the phase - of a non-Markovian process: for the even process, the truncation of ``0111`` is - ``111``, whose parity is unknown. So the length-``Lmax + 1`` suffix is tested - against the truncated suffix's state, and if its morph differs, it moves to the - alive state whose morph it matches best instead. ``resolve=False`` always truncates. - """ - history_to_state = {h: index for index in alive for h in states[index]} - edges: dict[int, dict[Any, dict[int, set[History]]]] = {} - for index in alive: - by_symbol: dict[Any, dict[int, set[History]]] = defaultdict(lambda: defaultdict(set)) - for history in states[index]: - for symbol, count in counts.next_counts.get(history, Counter()).items(): - if count == 0: - continue - extended = (*history, symbol) - if len(extended) <= Lmax: - target = history_to_state.get(extended) - else: - target = history_to_state.get(extended[1:]) - if ( - resolve - and counts.next_counts.get(extended) - and ( - target is None or morphs_differ(counts, states[target], {extended}, alpha=alpha, test=test) - ) - ): - best_score = float("inf") - for candidate in sorted(alive): - if morphs_differ(counts, states[candidate], {extended}, alpha=alpha, test=test): - continue - score = morph_test_score(counts, states[candidate], {extended}, test=test) - if score < best_score: - target, best_score = candidate, score - if target is not None: - by_symbol[symbol][target].add(history) - edges[index] = by_symbol - return edges - - -def _recurrent_states(edges: dict[int, dict[Any, dict[int, set[History]]]]) -> list[set[int]]: - """Closed communicating classes (with at least one edge) of the state graph.""" - import networkx as nx - - graph = nx.DiGraph() - graph.add_nodes_from(edges) - for source, by_symbol in edges.items(): - for targets in by_symbol.values(): - graph.add_edges_from((source, target) for target in targets) - condensed = nx.condensation(graph) - classes = [] - for node in condensed: - members = set(condensed.nodes[node]["members"]) - if condensed.out_degree(node) == 0 and graph.subgraph(members).number_of_edges() > 0: - classes.append(members) - return classes - - -def _suffix_determinize( - states: list[set[History]], - counts: SuffixCounts, - alive: set[int], - *, - Lmax: int, - alpha: float, - test: MorphTest, - resolve: bool = True, -) -> tuple[list[set[History]], set[int]]: - """Split alive states until each (state, symbol) pair has a single alive successor. - - Successors in pruned (transient) states are ignored, as in :cite:`Shalizi2002`. - """ - states = [set(h) for h in states] - alive = set(alive) - while True: - edges = _suffix_edges(states, counts, alive, Lmax=Lmax, alpha=alpha, test=test, resolve=resolve) - split = None - for index in sorted(alive): - for symbol in sorted(edges[index], key=repr): - if len(edges[index][symbol]) > 1: - split = (index, symbol) - break - if split: - break - if split is None: - return states, alive - index, symbol = split - groups = sorted(edges[index][symbol].values(), key=lambda g: (-len(g), sorted(map(repr, g)))) - for group in groups[1:]: - states[index] -= group - states.append(set(group)) - alive.add(len(states) - 1) - - -def _suffix_machine( - states: list[set[History]], - counts: SuffixCounts, - sequence: Sequence[Any], - alive: set[int], - *, - Lmax: int, - alpha: float, - test: MorphTest, - resolve: bool = True, -) -> EpsilonMachine: - """Build the ε-machine on the most-visited recurrent class of the alive states.""" - edges = _suffix_edges(states, counts, alive, Lmax=Lmax, alpha=alpha, test=test, resolve=resolve) - history_to_state = {h: index for index in alive for h in states[index]} - - visits: Counter[int] = Counter() - seq = tuple(sequence) - for t in range(len(seq) + 1): - for length in range(min(t, Lmax), -1, -1): - state = history_to_state.get(seq[t - length : t]) - if state is not None: - visits[state] += 1 - break - - classes = _recurrent_states(edges) - if not classes: - raise StochasticValidationError("no recurrent inferred states; the sample is too short for this Lmax") - keep = max(classes, key=lambda members: (sum(visits[s] for s in members), -min(members))) - - labels = {state: f"s{rank}" for rank, state in enumerate(sorted(keep))} - transitions = TransitionGraph() - for state in sorted(keep): - transitions.add_state(labels[state]) - for state in sorted(keep): - longest = max(len(h) for h in states[state]) - observed = _observed_counts_for_morph(counts, {h for h in states[state] if len(h) == longest}) - weights = { - symbol: (next(iter(targets)), float(observed.get(symbol, 0))) - for symbol, targets in edges[state].items() - if observed.get(symbol, 0) > 0 - } - if not weights: - weights = { - symbol: (next(iter(targets)), float(sum(len(h) for h in targets.values()))) - for symbol, targets in edges[state].items() - } - total = sum(weight for _, weight in weights.values()) - for symbol, (target, weight) in sorted(weights.items(), key=lambda item: repr(item[0])): - transitions.add_transition( - labels[state], labels[target], **{ATTR_PROB: weight / total, ATTR_EMISSION: symbol} - ) - - kept_visits = {state: visits[state] for state in keep if visits[state] > 0} - total_visits = float(sum(kept_visits.values())) - initial = ( - {labels[state]: count / total_visits for state, count in kept_visits.items()} - if total_visits > 0 - else {labels[min(keep)]: 1.0} - ) - machine = EpsilonMachine( - graph=transitions, - initial_distribution=initial, - observation_alphabet=frozenset(counts.alphabet), - ) - machine.validate() - return machine - - -def _suffix_reconstruct( - states: list[set[History]], - counts: SuffixCounts, - sequence: Sequence[Any], - *, - Lmax: int, - alpha: float, - test: MorphTest, -) -> EpsilonMachine: - """Prune transient states, determinize, and build the machine from homogeneous ``states``.""" - - def reconstruct(resolve: bool) -> EpsilonMachine: - everything = set(range(len(states))) - edges = _suffix_edges(states, counts, everything, Lmax=Lmax, alpha=alpha, test=test, resolve=resolve) - alive = set().union(*_recurrent_states(edges)) or everything - split, alive = _suffix_determinize(states, counts, alive, Lmax=Lmax, alpha=alpha, test=test, resolve=resolve) - return _suffix_machine(split, counts, sequence, alive, Lmax=Lmax, alpha=alpha, test=test, resolve=resolve) - - machine = reconstruct(resolve=True) - # Resolving truncated successors needs Lmax at least the synchronization length. When it - # is shorter, resolution can close off a state that never emits some observed symbol. - if {t.data[ATTR_EMISSION] for t in machine.transitions()} < set(sequence): - machine = reconstruct(resolve=False) - return machine - - -def suggest_lmax( - sequence: Sequence[Any], - *, - alpha: float = 0.01, - max_order: int | None = None, - method: Literal["exact", "chi2", "aic", "bic"] = "exact", - n_surrogates: int = 999, - seed: int = 0, -) -> int: - """A data-driven ``Lmax`` for :func:`cssr`: the estimated Markov order, at least 1. - - Orders ``0, 1, ...`` are tested against the next order with - :func:`dit.inference.select_markov_order`. The default ``"exact"`` method - compares the conditional block entropy against surrogates that preserve the - observed ``(order + 1)``-gram counts, which is valid at any sample size, unlike - the asymptotic chi-squared test :cite:`Pethel2014`. - - Parameters - ---------- - sequence - Observed symbols. - alpha - Significance level of each order test. - max_order - Largest order considered; by default the largest ``L`` whose - ``(L + 1)``-words are seen about 5 times each on average, at most 10. - method - ``"exact"`` or ``"chi2"`` (sequential tests), or ``"aic"`` / ``"bic"``. - n_surrogates - Surrogates per test for ``"exact"``. - seed - Seed for the surrogates, so the suggestion is reproducible. - - Notes - ----- - For a Markov source this recovers its order, which is the synchronization - length CSSR needs. A strictly sofic source (such as the even process) has - infinite Markov order, so the suggestion keeps growing with the sample; treat it - as a lower bound on the history length the data can support, not as the source's - synchronization length. - """ - import dit.inference - - select_markov_order = getattr(dit.inference, "select_markov_order", None) - if select_markov_order is None: # pragma: no cover - depends on the installed dit - raise ImportError("suggest_lmax requires a dit release with dit.inference.select_markov_order") - seq = [repr(symbol) for symbol in sequence] - if max_order is None: - k = max(2, len(set(seq))) - max_order = max(1, min(10, int(np.log(max(len(seq), 1) / 5) / np.log(k)) - 1)) - order = select_markov_order(seq, max_order, method=method, alpha=alpha, n_surrogates=n_surrogates, prng=seed) - return max(1, int(order)) - - -def cssr( - sequence: Sequence[Any], - *, - alphabet: Sequence[Any] | None = None, - Lmax: int | Literal["auto"] | None = None, - alpha: float = 0.01, - test: MorphTest = "g", - min_count: int = 5, - correction: Literal["bonferroni"] | None = None, -) -> EpsilonMachine: - """Reconstruct an ε-machine by Causal-State Splitting Reconstruction :cite:`Shalizi2004`. - - Suffixes are grown one symbol into the past up to length ``Lmax`` and grouped by - their next-symbol distributions (homogenization), then states are split until - every transition is deterministic (determinization). The result is restricted - to its most-visited closed class, so it is always a valid recurrent machine. - - Parameters - ---------- - sequence - Observed symbols. - alphabet - Symbol alphabet; defaults to the symbols in ``sequence``. - Lmax - Longest suffix considered; by default a third of ``log_k len(sequence)``, - between 1 and 10. It should be at least the synchronization length of the - source (for a Markov source, its order). Larger values run many more - significance tests, and some of them split states by chance. ``"auto"`` - uses :func:`suggest_lmax`, the Markov order estimated by exact tests. - alpha - Significance level of each morph-equality test. The worked example of - :cite:`Shalizi2002` uses 0.01; smaller values guard against spurious states - when ``Lmax`` is large. - test - ``"g"`` (G-test), ``"chi2"``, ``"tv"`` (total-variation threshold), or - ``"exact"`` (Monte Carlo exact G-test when expected counts are small; see - :func:`morphs_differ`). - min_count - Suffixes seen fewer than this many times are not tested or placed in a state. - correction - ``"bonferroni"`` divides ``alpha`` by the number of suffixes eligible for - testing, bounding the chance of any spurious split. CSSR decides each test - in light of earlier ones, so step-up procedures that control the false - discovery rate (Benjamini–Hochberg) do not apply directly. - - Notes - ----- - A process that is not exactly synchronizable (no finite past determines its - state, such as :func:`~sofic.examples.processes.ABC`) has no finite-``Lmax`` - reconstruction. CSSR then returns more states than the ε-machine, with an - entropy rate that approaches the true one from above as ``Lmax`` grows. - """ - seq = tuple(sequence) - if len(seq) < 2: - raise ValueError("sequence must contain at least two symbols") - alphabet_size = len(set(seq)) if alphabet is None else len(tuple(alphabet)) - if Lmax == "auto": - max_length = suggest_lmax(seq, alpha=alpha) - else: - max_length = Lmax if Lmax is not None else _cssr_default_lmax(len(seq), alphabet_size) - if max_length < 0: - raise ValueError("Lmax must be non-negative") - counts = SuffixCounts.from_sequence(seq, alphabet=alphabet, max_length=max_length + 1) - if correction == "bonferroni": - alpha = _bonferroni_alpha(counts, alpha, max_length=max_length, min_count=min_count) - elif correction is not None: - raise ValueError(f"unknown correction {correction!r}") - - homogeneous = _suffix_homogenize(counts, Lmax=max_length, alpha=alpha, test=test, min_count=min_count) - return _suffix_reconstruct(homogeneous, counts, seq, Lmax=max_length, alpha=alpha, test=test) - - -#: Significance level used by subtree merging when ``delta = 0`` and to resolve truncated successors. -_SUBTREE_ALPHA = 0.01 - - -def _morph_distance( - counts: SuffixCounts, - left: History, - right: History, - *, - delta: float, -) -> float: - left_morph = counts.morph(left) - right_morph = counts.morph(right) - return 0.5 * sum(abs(left_morph[s] - right_morph[s]) for s in counts.alphabet) - - -def _morphs_equivalent( - counts: SuffixCounts, - left: History, - right: History, - *, - delta: float, - alpha: float = _SUBTREE_ALPHA, - test: MorphTest = "g", -) -> bool: - if delta > 0.0: - return _morph_distance(counts, left, right, delta=delta) <= delta - return not morphs_differ(counts, {left}, {right}, alpha=alpha, test=test) - - -def _cluster_histories_by_morph( - counts: SuffixCounts, - histories: set[History], - *, - delta: float, - alpha: float = _SUBTREE_ALPHA, - test: MorphTest = "g", -) -> dict[int, set[History]]: - parent: dict[History, History] = {history: history for history in histories} - - def find(history: History) -> History: - root = history - while parent[root] != root: - parent[root] = parent[parent[root]] - root = parent[root] - return root - - def union(left: History, right: History) -> None: - left_root = find(left) - right_root = find(right) - if left_root != right_root: - parent[right_root] = left_root - - history_list = sorted(histories) - for index, left in enumerate(history_list): - for right in history_list[index + 1 :]: - if _morphs_equivalent(counts, left, right, delta=delta, alpha=alpha, test=test): - union(left, right) - - clusters: dict[History, set[History]] = defaultdict(set) - for history in histories: - clusters[find(history)].add(history) - - states: dict[int, set[History]] = {} - for state_id, (_root, members) in enumerate(clusters.items()): - states[state_id] = set(members) - return states - - -def subtree_merge( - sequence: Sequence[Any], - *, - L: int | Literal["auto"], - delta: float = 0.0, - alphabet: Sequence[Any] | None = None, - alpha: float = _SUBTREE_ALPHA, - test: MorphTest = "g", - correction: Literal["bonferroni"] | None = None, -) -> EpsilonMachine: - """Reconstruct an ε-machine by merging depth-``L`` subtrees (Crutchfield--Young). - - Histories up to length ``L`` are clustered by next-symbol distribution: within - total-variation distance ``delta``, or, when ``delta = 0``, unless a morph test - (``test``, at level ``alpha``) tells them apart. The clusters are then - determinized as in :func:`cssr`. - - ``L="auto"`` uses :func:`suggest_lmax`. ``correction="bonferroni"`` divides - ``alpha`` by the number of history pairs compared, so that no pair is split - apart by chance; since a rejected test *separates* histories, this makes the - reconstruction more conservative (fewer states). - """ - if L == "auto": - L = suggest_lmax(sequence, alpha=alpha) - if L < 0: - raise ValueError("L must be non-negative") - seq = tuple(sequence) - if len(seq) < 2: - raise ValueError("sequence must contain at least two symbols") - counts = SuffixCounts.from_sequence(seq, alphabet=alphabet, max_length=L + 1) - - histories = {history for history in counts.history_counts if len(history) <= L} - histories.add(()) - if correction == "bonferroni": - alpha /= max(1, len(histories) * (len(histories) - 1) // 2) - elif correction is not None: - raise ValueError(f"unknown correction {correction!r}") - states = list(_cluster_histories_by_morph(counts, histories, delta=delta, alpha=alpha, test=test).values()) - - return _suffix_reconstruct(states, counts, seq, Lmax=L, alpha=alpha, test=test) - - -def spectral( - sequences: Iterable[Any] | None = None, - *, - word_probability: Callable[[Sequence[Any]], float] | None = None, - alphabet: Sequence[Any] | None = None, - rank: int | None = None, - prefix_length: int = 3, - suffix_length: int | None = None, - singular_value_threshold: float = 1e-3, - min_singular_value: float = 1e-12, - max_states: int = 10_000, -) -> EpsilonMachine: - """Reconstruct an ε-machine by spectral learning then mixed-state extraction. - - Learns a weighted finite automaton / observable-operator model from block - statistics :cite:`Balle2014,Hsu2012`, then extracts causal states as the - mixed states of those operators :cite:`Ellison2009`. When the learned - operators are non-negative this is a Mealy projection followed by - :meth:`~sofic.generators.epsilon_machine.EpsilonMachine.from_hmm`; signed - operators use mixed-state enumeration rather than a clustering heuristic. - - Parameters - ---------- - sequences - A single observed realization or an iterable of realizations. Ignored - when ``word_probability`` is given. - word_probability - Optional exact block-probability function ``f(word) -> float``. - ``alphabet`` is then required. - alphabet - Observation alphabet. Inferred from ``sequences`` when omitted. - rank - Number of latent states. When ``None`` the rank is chosen from the - Hankel singular-value spectrum. - prefix_length, suffix_length - Maximum lengths of the prefix and suffix bases. ``suffix_length`` - defaults to ``prefix_length``. - singular_value_threshold, min_singular_value - Cutoffs for automatic rank selection; see - :func:`~sofic.inference.spectral.learn_spectral_wfa`. - max_states - Safety cap on enumerated mixed states. - """ - from sofic.inference.spectral import learn_spectral_wfa, project_to_epsilon_machine - - model = learn_spectral_wfa( - sequences, - word_probability=word_probability, - alphabet=alphabet, - rank=rank, - prefix_length=prefix_length, - suffix_length=suffix_length, - singular_value_threshold=singular_value_threshold, - min_singular_value=min_singular_value, - ) - return project_to_epsilon_machine(model, max_states=max_states) diff --git a/sofic/generators/epsilon_machine.py b/sofic/generators/epsilon_machine.py index 0372e26..b430963 100644 --- a/sofic/generators/epsilon_machine.py +++ b/sofic/generators/epsilon_machine.py @@ -66,22 +66,22 @@ def from_sequence( ``"spectral"`` for Hankel-SVD learning followed by mixed-state extraction. **kwargs - Forwarded to :func:`~sofic.generators.epsilon_inference.cssr`, - :func:`~sofic.generators.epsilon_inference.subtree_merge`, or - :func:`~sofic.generators.epsilon_inference.spectral`. + Forwarded to :func:`~sofic.inference.cssr.learn_epsilon_machine_cssr`, + :func:`~sofic.inference.cssr.learn_epsilon_machine_subtree`, or + :func:`~sofic.inference.spectral.learn_epsilon_machine_spectral`. """ if method == "cssr": - from sofic.generators.epsilon_inference import cssr + from sofic.inference.cssr.process import learn_epsilon_machine_cssr - return cssr(sequence, **kwargs) + return learn_epsilon_machine_cssr(sequence, **kwargs) if method == "subtree": - from sofic.generators.epsilon_inference import subtree_merge + from sofic.inference.cssr.subtree import learn_epsilon_machine_subtree - return subtree_merge(sequence, **kwargs) + return learn_epsilon_machine_subtree(sequence, **kwargs) if method == "spectral": - from sofic.generators.epsilon_inference import spectral + from sofic.inference.spectral import learn_epsilon_machine_spectral - return spectral(sequence, **kwargs) + return learn_epsilon_machine_spectral(sequence, **kwargs) raise ValueError(f"unknown inference method {method!r}") def copy(self) -> Self: diff --git a/sofic/generators/epsilon_transducer.py b/sofic/generators/epsilon_transducer.py index c955e1c..3d091da 100644 --- a/sofic/generators/epsilon_transducer.py +++ b/sofic/generators/epsilon_transducer.py @@ -41,9 +41,9 @@ class EpsilonTransducer(MealyMachine): Examples -------- - >>> from sofic.examples.processes import BinaryChannel + >>> from sofic.examples.processes import binary_channel >>> from sofic import EpsilonTransducer - >>> channel = BinaryChannel(0.1, 0.2) + >>> channel = binary_channel(0.1, 0.2) >>> eps = EpsilonTransducer.from_channel(channel) >>> eps.is_unifilar() True @@ -128,9 +128,9 @@ def from_paired_sequences( **kwargs: Any, ) -> EpsilonTransducer: """Reconstruct an ε-transducer from paired input/output sequences via transCSSR.""" - from sofic.generators.epsilon_transducer_inference import transcssr + from sofic.inference.cssr.transducer import learn_epsilon_transducer_cssr - return transcssr(inputs, outputs, **kwargs) + return learn_epsilon_transducer_cssr(inputs, outputs, **kwargs) # -- channel measures ----------------------------------------------------- diff --git a/sofic/generators/hmm_inference.py b/sofic/generators/hmm_inference.py deleted file mode 100644 index f83728a..0000000 --- a/sofic/generators/hmm_inference.py +++ /dev/null @@ -1,783 +0,0 @@ -"""Inference for hidden Markov models. - -Forward/backward/Viterbi decoding and sampling, plus the Cappe, Moulines & -Ryden (2005) toolbox: fixed-interval smoothing (one- and two-slice marginals), -Baum-Welch EM parameter re-estimation, and the score / observed information via -the Fisher and Louis identities. -""" - -from __future__ import annotations - -from collections import defaultdict -from collections.abc import Hashable, Iterable, Sequence -from typing import Any - -import numpy as np - -from sofic.generators.base import HiddenMarkovModel -from sofic.graph import ATTR_EMISSION, ATTR_PROB - - -def _as_mealy_hmm(hmm: HiddenMarkovModel) -> Any: - """Return a Mealy-style representation through the HMM representation hook.""" - return hmm.to_mealy() - - -def _emission_transition_tensors_from_mealy( - hmm: Any, -) -> tuple[np.ndarray, dict[Any, np.ndarray]]: - """Return initial vector ``pi`` and symbol -> joint transition matrices. - - Symbols are keyed in a fixed (``repr``-sorted) order, so seeded sampling is - reproducible across interpreter runs. A model without an initial - distribution starts from its stationary distribution. - """ - from sofic.generators.prob import as_prob, has_symbolic, zeros - - idx = hmm.reindex() - n = len(idx) - edge_probs = [transition.data.get(ATTR_PROB, 0.0) for transition in hmm.transitions()] - init_probs = list(hmm.initial_distribution.values()) - symbolic = has_symbolic(edge_probs) or has_symbolic(init_probs) - - pi = zeros((n,), symbolic=symbolic) - if hmm.initial_distribution: - for state, mass in hmm.initial_distribution.items(): - pi[idx.index(state)] = as_prob(mass) - elif n: - pi = np.asarray(hmm.stationary_distribution(), dtype=object if symbolic else float) - - emissions = {transition.data.get(ATTR_EMISSION) for transition in hmm.transitions()} - {None} - symbols = sorted(set(hmm.observation_alphabet) | emissions, key=repr) - joint: dict[Any, np.ndarray] = {symbol: zeros((n, n), symbolic=symbolic) for symbol in symbols} - - for transition in hmm.transitions(): - emission = transition.data.get(ATTR_EMISSION) - if emission is None: - continue - i = idx.index(transition.source) - j = idx.index(transition.target) - joint[emission][i, j] = as_prob(joint[emission][i, j]) + as_prob(transition.data.get(ATTR_PROB, 0.0)) - return pi, joint - - -def _emission_transition_tensors( - hmm: HiddenMarkovModel, -) -> tuple[np.ndarray, dict[Any, np.ndarray]]: - """Return initial vector ``pi`` and symbol -> joint transition matrices.""" - return _emission_transition_tensors_from_mealy(_as_mealy_hmm(hmm)) - - -def _limit_distribution_from_initial(pi_initial: np.ndarray, transition: np.ndarray) -> np.ndarray | None: - """Return the limiting occupation law of ``pi_initial`` under ``transition``. - - On reducible chains the left-eigenvector stationary law is not unique; the - process measure is the limit reached from the model's initial distribution. - """ - pi = np.asarray(pi_initial, dtype=float).copy() - total = float(pi.sum()) - if total <= 0.0: - return None - pi /= total - matrix = np.asarray(transition, dtype=float) - n = len(pi) - for _ in range(max(100, 20 * n)): - nxt = pi @ matrix - mass = float(nxt.sum()) - if mass <= 0.0: - return None - nxt /= mass - if np.allclose(nxt, pi, rtol=1e-12, atol=1e-14): - pi = nxt - break - pi = nxt - pi[np.isclose(pi, 0.0, atol=1e-15)] = 0.0 - mass = float(pi.sum()) - if mass <= 0.0: - return None - return pi / mass - - -def _stationary_emission_tensors( - hmm: HiddenMarkovModel, -) -> tuple[np.ndarray, dict[Any, np.ndarray]]: - """Return the stationary state law and symbol -> joint transition matrices. - - Block/word statistics of a *stationary* process must weight the initial state - by the stationary distribution, not by the model's (possibly transient) - ``initial_distribution``. The stationary vector is recovered directly from the - summed emission-transition matrices so it stays aligned with ``joint``'s state - indexing. - - When the chain is reducible (multiple absorbing classes), the eigenvector - stationary law is not unique — prefer the limiting occupation reached from - ``initial_distribution``. Fall back to the eigenvector solution, then to the - initial vector, only when the limit cannot be formed. - """ - from sofic.generators.prob import zeros - from sofic.generators.stationary import stationary_distribution_from_transition - - pi_initial, joint = _emission_transition_tensors(hmm) - n = len(pi_initial) - if n == 0: - return pi_initial, joint - symbolic = pi_initial.dtype == object or any(matrix.dtype == object for matrix in joint.values()) - transition = zeros((n, n), symbolic=symbolic) - for matrix in joint.values(): - transition = transition + matrix - if not symbolic: - limited = _limit_distribution_from_initial(pi_initial, transition) - if limited is not None and np.allclose(limited @ transition, limited, rtol=1e-8, atol=1e-10): - return limited, joint - try: - pi = stationary_distribution_from_transition(transition) - except Exception: - pi = pi_initial - return pi, joint - - -def _forward_scaled( - pi: np.ndarray, - joint: dict[Any, np.ndarray], - obs: list[Any], -) -> tuple[np.ndarray, np.ndarray]: - """Return per-step-normalized forward messages and log scaling factors. - - ``alpha_hat[t]`` sums to one; ``log P(obs) = log_scales.sum()``. A ``-inf`` - entry in ``log_scales`` marks an impossible step. Normalizing each step avoids - the underflow that makes the raw forward product vanish for long sequences. - """ - n = len(pi) - alpha_hat = np.zeros((len(obs) + 1, n), dtype=float) - log_scales = np.zeros(len(obs) + 1, dtype=float) - total0 = float(pi.sum()) - if total0 <= 0.0: - log_scales[0] = -np.inf - return alpha_hat, log_scales - alpha_hat[0] = pi / total0 - log_scales[0] = float(np.log(total0)) - for t, symbol in enumerate(obs): - matrix = joint.get(symbol) - if matrix is None: - log_scales[t + 1] = -np.inf - continue - row = alpha_hat[t] @ matrix - scale = float(row.sum()) - if scale <= 0.0: - log_scales[t + 1] = -np.inf - continue - alpha_hat[t + 1] = row / scale - log_scales[t + 1] = float(np.log(scale)) - return alpha_hat, log_scales - - -def forward(hmm: HiddenMarkovModel, observations: Sequence[Any], *, scaled: bool = False) -> np.ndarray: - """Return forward messages ``alpha[t, s]`` for ``len(observations)+1`` rows. - - With ``scaled=True`` each row is normalized to sum to one (the numerically - stable message used for posteriors); otherwise the raw messages are returned. - """ - pi, joint = _emission_transition_tensors(hmm) - obs = list(observations) - if scaled: - alpha_hat, _log_scales = _forward_scaled(pi, joint, obs) - return alpha_hat - n = len(pi) - alpha = np.zeros((len(obs) + 1, n), dtype=float) - alpha[0] = pi - for t, symbol in enumerate(obs): - matrix = joint.get(symbol) - if matrix is None: - alpha[t + 1] = 0.0 - else: - alpha[t + 1] = alpha[t] @ matrix - return alpha - - -def backward(hmm: HiddenMarkovModel, observations: Sequence[Any], *, scaled: bool = False) -> np.ndarray: - """Return backward messages ``beta[t, s]`` for ``len(observations)+1`` rows. - - With ``scaled=True`` each row is normalized to sum to one. The smoothed - posterior is then ``normalize(alpha_hat[t] * beta_hat[t])`` (the per-row - scaling constants cancel on renormalization). - """ - _, joint = _emission_transition_tensors(hmm) - n = next(iter(joint.values())).shape[0] if joint else len(_as_mealy_hmm(hmm).reindex()) - obs = list(observations) - beta = np.zeros((len(obs) + 1, n), dtype=float) - beta[len(obs)] = 1.0 - for t in range(len(obs) - 1, -1, -1): - matrix = joint.get(obs[t]) - if matrix is None: - beta[t] = 0.0 - else: - beta[t] = matrix @ beta[t + 1] - if scaled: - total = float(beta[t].sum()) - if total > 0.0: - beta[t] = beta[t] / total - return beta - - -def _backward_scaled(joint: dict[Any, np.ndarray], obs: list[Any], n_states: int) -> np.ndarray: - """Per-row-normalized backward messages from precomputed transition tensors. - - ``beta_hat[t]`` sums to one; the per-row scaling constants cancel against the - forward scaling when the smoothed posterior is renormalized. Shares tensors - with the forward pass so smoothing and EM avoid recomputing them. - """ - beta = np.zeros((len(obs) + 1, n_states), dtype=float) - beta[len(obs)] = 1.0 - for t in range(len(obs) - 1, -1, -1): - matrix = joint.get(obs[t]) - row = beta[t + 1] if matrix is None else matrix @ beta[t + 1] - beta[t] = 0.0 if matrix is None else row - total = float(beta[t].sum()) - if total > 0.0: - beta[t] = beta[t] / total - return beta - - -def log_likelihood(hmm: HiddenMarkovModel, observations: Sequence[Any]) -> float: - """Natural-log likelihood ``log P(observations)``. - - Uses the per-step-scaled forward recursion so the result stays finite for long - sequences instead of underflowing to ``-inf``. - """ - pi, joint = _emission_transition_tensors(hmm) - _alpha_hat, log_scales = _forward_scaled(pi, joint, list(observations)) - if not np.all(np.isfinite(log_scales)): - return float("-inf") - return float(log_scales.sum()) - - -def smooth(hmm: HiddenMarkovModel, observations: Sequence[Any]) -> np.ndarray: - r"""Return fixed-interval smoothed marginals ``gamma[t, s]``. - - ``gamma[t, s] = P(X_t = s \mid Y_{0:n-1})`` for ``t = 0, ..., n`` (there are - ``n + 1`` hidden states behind ``n`` edge emissions). Computed as the - per-row-renormalized product of the scaled forward and backward messages, the - forward-backward smoother of Cappe, Moulines & Ryden (2005, Section 3.2). - Rows for observation sequences of zero probability are returned as zeros. - """ - pi, joint = _emission_transition_tensors(hmm) - obs = list(observations) - n_states = len(pi) - alpha_hat, log_scales = _forward_scaled(pi, joint, obs) - if not np.all(np.isfinite(log_scales)): - return np.zeros((len(obs) + 1, n_states), dtype=float) - beta_hat = _backward_scaled(joint, obs, n_states) - gamma = alpha_hat * beta_hat - row_sums = gamma.sum(axis=1, keepdims=True) - with np.errstate(invalid="ignore", divide="ignore"): - gamma = np.where(row_sums > 0.0, gamma / row_sums, 0.0) - return gamma - - -def two_slice_marginals(hmm: HiddenMarkovModel, observations: Sequence[Any]) -> np.ndarray: - r"""Return two-slice smoothed marginals ``xi[t, i, j]``. - - ``xi[t, i, j] = P(X_t = i, X_{t+1} = j \mid Y_{0:n-1})`` for ``t = 0, ..., n-1``, - where the transition at index ``t`` emits ``Y_t`` (Cappe, Moulines & Ryden, - 2005, Section 3.2). Marginalizing over ``j`` recovers ``gamma[t]`` for - ``t < n``. Returns an all-zero tensor for zero-probability sequences. - """ - pi, joint = _emission_transition_tensors(hmm) - obs = list(observations) - n_states = len(pi) - xi = np.zeros((len(obs), n_states, n_states), dtype=float) - alpha_hat, log_scales = _forward_scaled(pi, joint, obs) - if not np.all(np.isfinite(log_scales)): - return xi - beta_hat = _backward_scaled(joint, obs, n_states) - for t, symbol in enumerate(obs): - matrix = joint.get(symbol) - if matrix is None: - continue - block = alpha_hat[t][:, None] * matrix * beta_hat[t + 1][None, :] - total = float(block.sum()) - if total > 0.0: - xi[t] = block / total - return xi - - -def _expected_edge_counts( - pi: np.ndarray, - joint: dict[Any, np.ndarray], - obs: list[Any], -) -> tuple[dict[tuple[int, Any, int], float], np.ndarray, np.ndarray, float]: - r"""Expected sufficient statistics for one observation sequence. - - Returns ``(edge_counts, source_totals, gamma0, loglik)`` where - - - ``edge_counts[(i, symbol, j)]`` is - :math:`\sum_t P(X_t = i, Y_t = symbol, X_{t+1} = j \mid Y)`, the expected - number of uses of edge ``i --symbol--> j``; - - ``source_totals[i] = \sum_{t=0}^{n-1} P(X_t = i \mid Y)`` is the expected - number of transitions out of state ``i`` (the Baum-Welch denominator); - - ``gamma0`` is the smoothed marginal of the initial state ``X_0``; - - ``loglik`` is the natural-log likelihood of the sequence. - - Only edges present in ``joint`` (structural support) receive mass, so the - statistics preserve the model topology. - """ - n_states = len(pi) - alpha_hat, log_scales = _forward_scaled(pi, joint, obs) - if not np.all(np.isfinite(log_scales)): - return {}, np.zeros(n_states), np.zeros(n_states), float("-inf") - beta_hat = _backward_scaled(joint, obs, n_states) - edge_counts: dict[tuple[int, Any, int], float] = {} - source_totals = np.zeros(n_states, dtype=float) - for t, symbol in enumerate(obs): - matrix = joint.get(symbol) - if matrix is None: - continue - block = alpha_hat[t][:, None] * matrix * beta_hat[t + 1][None, :] - total = float(block.sum()) - if total <= 0.0: - continue - block = block / total - source_totals += block.sum(axis=1) - for i, j in np.argwhere(block > 0.0): - key = (int(i), symbol, int(j)) - edge_counts[key] = edge_counts.get(key, 0.0) + float(block[i, j]) - g0 = alpha_hat[0] * beta_hat[0] - s0 = float(g0.sum()) - gamma0 = g0 / s0 if s0 > 0.0 else np.zeros(n_states) - return edge_counts, source_totals, gamma0, float(log_scales.sum()) - - -def _as_sequence_list(sequences: Iterable[Any]) -> list[list[Any]]: - """Normalize ``sequences`` to a list of observation sequences. - - Accepts either a single flat observation sequence (e.g. ``[0, 1, 0]``) or an - iterable of sequences (e.g. ``[[0, 1], [1, 0]]``). A single sequence is - detected when its first element is not itself a non-string sequence. - """ - seqs = list(sequences) - if not seqs: - return [] - first = seqs[0] - if isinstance(first, (list, tuple)) and not isinstance(first, (str, bytes)): - return [list(seq) for seq in seqs] - return [seqs] - - -def baum_welch( - hmm: HiddenMarkovModel, - sequences: Iterable[Any], - *, - max_iter: int = 100, - tol: float = 1e-6, - estimate_initial: bool = True, - n_restarts: int = 1, - rng: np.random.Generator | int | None = None, - return_restarts: bool = False, -) -> tuple[Any, list[float]] | tuple[Any, list[float], list[float]]: - r"""Fit HMM parameters by Baum-Welch (EM) expectation-maximization. - - Re-estimates the Mealy joint edge law - :math:`A_o[i, j] = P(X_{t+1} = j, O = o \mid X_t = i)` and (optionally) the - initial distribution from data, holding the transition-graph topology fixed: - structurally absent edges receive zero expected count and stay absent, so the - fitted model generates the same sofic shift as ``hmm``. This is the EM - algorithm for probabilistic functions of finite Markov chains of Baum, Petrie, - Soules & Weiss and Cappe, Moulines & Ryden (2005, Chapter 10); see also - Rabiner (1989). - - ``sequences`` may be a single observation sequence or an iterable of - sequences (several sequences are needed to identify the initial distribution; - Cappe, Moulines & Ryden, 2005, Section 10.1). Unifilarity is *not* preserved, - so the fit is returned as a plain :class:`~sofic.generators.mealy.MealyHMM`. - - Returns ``(fitted_model, loglik_trace)`` where ``loglik_trace`` is the - non-decreasing sequence of total natural-log likelihoods observed before each - parameter update. - - EM converges to a local maximum of the likelihood. With ``n_restarts > 1`` the - first run starts from ``hmm``'s parameters and each further run from edge laws - drawn uniformly (Dirichlet(1)) over each state's structurally allowed edges; - the fit with the highest final log-likelihood is returned. Pass - ``return_restarts=True`` to also get every run's final log-likelihood, which - shows whether near-equal optima exist. - """ - from sofic.generators.mealy import MealyHMM - - mealy = hmm.to_mealy() - idx = mealy.reindex() - n_states = len(idx) - states = [idx.state(i) for i in range(n_states)] - alphabet = frozenset(mealy.observation_alphabet) - seqs = _as_sequence_list(sequences) - - pi, joint = _emission_transition_tensors_from_mealy(mealy) - support = { - (i, symbol, j) - for symbol, matrix in joint.items() - for i in range(n_states) - for j in range(n_states) - if matrix[i, j] > 0.0 - } - - if n_restarts < 1: - raise ValueError("n_restarts must be at least 1") - generator = rng if isinstance(rng, np.random.Generator) else np.random.default_rng(rng) - runs = [] - for restart in range(n_restarts): - start_joint = joint if restart == 0 else _random_edge_law(joint, support, n_states, generator) - runs.append( - _baum_welch_run(pi, start_joint, seqs, max_iter=max_iter, tol=tol, estimate_initial=estimate_initial) - ) - finals = [trace[-1] if trace else float("-inf") for _pi, _joint, trace in runs] - pi, joint, loglik_trace = runs[int(np.argmax(finals))] - - fitted = MealyHMM( - initial_distribution={states[i]: float(pi[i]) for i in range(n_states) if pi[i] > 0.0}, - observation_alphabet=alphabet, - ) - for state in states: - fitted.graph.add_state(state) - for i, symbol, j in sorted(support, key=lambda edge: (edge[0], str(edge[1]), edge[2])): - prob = float(joint[symbol][i, j]) - if prob > 0.0: - fitted.add_transition(states[i], states[j], symbol, prob) - fitted.validate() - if return_restarts: - return fitted, loglik_trace, finals - return fitted, loglik_trace - - -def _random_edge_law( - joint: dict[Any, np.ndarray], - support: set[tuple[int, Any, int]], - n_states: int, - rng: np.random.Generator, -) -> dict[Any, np.ndarray]: - """Edge laws drawn uniformly over each state's allowed ``(symbol, target)`` edges.""" - new_joint = {symbol: np.zeros((n_states, n_states), dtype=float) for symbol in joint} - for i in range(n_states): - edges = sorted(((symbol, j) for (source, symbol, j) in support if source == i), key=lambda e: (str(e[0]), e[1])) - if not edges: - continue - weights = rng.dirichlet(np.ones(len(edges))) - for (symbol, j), weight in zip(edges, weights, strict=True): - new_joint[symbol][i, j] = weight - return new_joint - - -def _baum_welch_run( - pi: np.ndarray, - joint: dict[Any, np.ndarray], - seqs: list[Any], - *, - max_iter: int, - tol: float, - estimate_initial: bool, -) -> tuple[np.ndarray, dict[Any, np.ndarray], list[float]]: - """One EM run from ``(pi, joint)``; returns the final parameters and trace.""" - n_states = len(pi) - loglik_trace: list[float] = [] - prev_ll: float | None = None - for _iteration in range(max_iter): - total_edge_counts: dict[tuple[int, Any, int], float] = defaultdict(float) - total_source = np.zeros(n_states, dtype=float) - gamma0_sum = np.zeros(n_states, dtype=float) - total_ll = 0.0 - for obs in seqs: - edge_counts, source_totals, gamma0, loglik = _expected_edge_counts(pi, joint, obs) - if not np.isfinite(loglik): - continue - for key, value in edge_counts.items(): - total_edge_counts[key] += value - total_source += source_totals - gamma0_sum += gamma0 - total_ll += loglik - loglik_trace.append(total_ll) - if prev_ll is not None and abs(total_ll - prev_ll) < tol: - break - prev_ll = total_ll - - new_joint = {symbol: np.zeros((n_states, n_states), dtype=float) for symbol in joint} - for (i, symbol, j), count in total_edge_counts.items(): - if total_source[i] > 0.0: - new_joint[symbol][i, j] = count / total_source[i] - for i in range(n_states): - if total_source[i] <= 0.0: - for symbol in joint: - new_joint[symbol][i, :] = joint[symbol][i, :] - joint = new_joint - if estimate_initial: - mass = float(gamma0_sum.sum()) - if mass > 0.0: - pi = gamma0_sum / mass - return pi, joint, loglik_trace - - -def score(hmm: HiddenMarkovModel, observations: Sequence[Any]) -> dict[tuple[Hashable, Any, Hashable], float]: - r"""Return the score (gradient of the log-likelihood) via the Fisher identity. - - For each edge ``i --o--> j``, returns - :math:`\partial \log P(Y) / \partial A_o[i, j] = E[N_{i,o,j} \mid Y] / A_o[i, j]`, - where ``N`` is the (unobserved) edge-use count. This is Fisher's identity, - ``\nabla \log L(\theta) = E[\nabla \log f(X, Y; \theta) \mid Y]`` (Cappe, - Moulines & Ryden, 2005, Section 10.2.3), evaluated in the raw (unconstrained) - joint-edge parameters. Keys are ``(source, symbol, target)`` state labels. - """ - mealy = hmm.to_mealy() - idx = mealy.reindex() - pi, joint = _emission_transition_tensors_from_mealy(mealy) - edge_counts, _source_totals, _gamma0, loglik = _expected_edge_counts(pi, joint, list(observations)) - if not np.isfinite(loglik): - raise ValueError("observations have zero probability under the model; score is undefined") - result: dict[tuple[Hashable, Any, Hashable], float] = {} - n_states = len(pi) - for symbol, matrix in joint.items(): - for i in range(n_states): - for j in range(n_states): - prob = float(matrix[i, j]) - if prob > 0.0: - count = edge_counts.get((i, symbol, j), 0.0) - result[(idx.state(i), symbol, idx.state(j))] = count / prob - return result - - -def _free_parameterization( - joint: dict[Any, np.ndarray], - n_states: int, -) -> tuple[list[tuple[int, Any, int]], list[tuple[int, Any, int]], list[int]]: - """Build the free multinomial parameterization of the joint edge law. - - Each source state whose outgoing edges number ``k >= 2`` contributes ``k - 1`` - free parameters (its last edge in canonical order is the reference). Returns - ``(free_edges, reference_by_param, source_by_param)``: the edge for each free - parameter, the reference edge of its source block, and the source-state index. - """ - free_edges: list[tuple[int, Any, int]] = [] - reference_by_param: list[tuple[int, Any, int]] = [] - source_by_param: list[int] = [] - for i in range(n_states): - out_edges = sorted( - ((i, symbol, j) for symbol, matrix in joint.items() for j in range(n_states) if matrix[i, j] > 0.0), - key=lambda edge: (str(edge[1]), edge[2]), - ) - if len(out_edges) < 2: - continue - reference = out_edges[-1] - for edge in out_edges[:-1]: - free_edges.append(edge) - reference_by_param.append(reference) - source_by_param.append(i) - return free_edges, reference_by_param, source_by_param - - -def observed_information(hmm: HiddenMarkovModel, observations: Sequence[Any]) -> np.ndarray: - r"""Return the observed information matrix via Louis' identity. - - The observed information ``J = -\partial^2 \log L / \partial\theta^2`` for the - free multinomial parameters of the joint edge law is obtained from Louis' - (1982) identity, - - .. math:: J = E[-\partial^2 \ell_c \mid Y] - \operatorname{Cov}(\partial \ell_c \mid Y), - - where :math:`\ell_c` is the complete-data log-likelihood (Cappe, Moulines & - Ryden, 2005, Section 10.2.3). The complete-data information ``B`` follows from - the expected edge counts; the conditional covariance of the complete-data - score is computed exactly by a forward smoothing recursion for the first and - second moments of the additive score functional. The matrix is ordered by - :func:`free_parameter_labels`; an empty ``(0, 0)`` matrix is returned when the - model has no free parameters. - """ - mealy = hmm.to_mealy() - pi, joint = _emission_transition_tensors_from_mealy(mealy) - obs = list(observations) - n_states = len(pi) - - free_edges, reference_by_param, source_by_param = _free_parameterization(joint, n_states) - d = len(free_edges) - if d == 0: - return np.zeros((0, 0), dtype=float) - - edge_counts, _source_totals, _gamma0, loglik = _expected_edge_counts(pi, joint, obs) - if not np.isfinite(loglik): - raise ValueError("observations have zero probability under the model; information is undefined") - - prob_of = {edge: float(joint[edge[1]][edge[0], edge[2]]) for edge in set(free_edges) | set(reference_by_param)} - - # Complete-data information B = E[-d^2 l_c | Y], block-diagonal by source state. - complete_information = np.zeros((d, d), dtype=float) - for p in range(d): - ref_p = reference_by_param[p] - count_ref = edge_counts.get(ref_p, 0.0) - ref_term = count_ref / prob_of[ref_p] ** 2 - for q in range(d): - if source_by_param[p] != source_by_param[q]: - continue - value = ref_term - if p == q: - edge_p = free_edges[p] - value += edge_counts.get(edge_p, 0.0) / prob_of[edge_p] ** 2 - complete_information[p, q] = value - - # Per-transition score contribution s(edge) as a d-vector (sparse per source block). - edge_score: dict[tuple[int, Any, int], np.ndarray] = {} - for p, edge in enumerate(free_edges): - edge_score.setdefault(edge, np.zeros(d))[p] += 1.0 / prob_of[edge] - for p, ref in enumerate(reference_by_param): - edge_score.setdefault(ref, np.zeros(d))[p] += -1.0 / prob_of[ref] - zero_d = np.zeros(d) - - # Forward smoothing recursion for E[S | Y] and E[S S^T | Y] of the additive - # complete-data score functional S = sum_t s(edge_t). - alpha_hat, _log_scales = _forward_scaled(pi, joint, obs) - first = np.zeros((n_states, d), dtype=float) - second = np.zeros((n_states, d, d), dtype=float) - for t, symbol in enumerate(obs): - matrix = joint.get(symbol) - if matrix is None: - continue - weight = alpha_hat[t][:, None] * matrix # weight[i, k] = P(X_t=i, X_{t+1}=k, Y_t | Y_{0:t-1}) - denom = weight.sum(axis=0) - new_first = np.zeros((n_states, d), dtype=float) - new_second = np.zeros((n_states, d, d), dtype=float) - for k in range(n_states): - if denom[k] <= 0.0: - continue - for i in range(n_states): - if weight[i, k] <= 0.0: - continue - retro = weight[i, k] / denom[k] # P(X_t=i | X_{t+1}=k, Y_{0:t}) - s_vec = edge_score.get((i, symbol, k), zero_d) - first_i = first[i] - combined = first_i + s_vec - new_first[k] += retro * combined - cross = np.outer(first_i, s_vec) - new_second[k] += retro * (second[i] + cross + cross.T + np.outer(s_vec, s_vec)) - first, second = new_first, new_second - - phi_final = alpha_hat[len(obs)] - expected_score = phi_final @ first - expected_outer = np.einsum("k,kpq->pq", phi_final, second) - score_covariance = expected_outer - np.outer(expected_score, expected_score) - return complete_information - score_covariance - - -def free_parameter_labels(hmm: HiddenMarkovModel) -> list[tuple[Hashable, Any, Hashable]]: - """Return the ``(source, symbol, target)`` label for each free parameter. - - The order matches the rows and columns of :func:`observed_information` and the - entries of :func:`standard_errors`. - """ - mealy = hmm.to_mealy() - idx = mealy.reindex() - _pi, joint = _emission_transition_tensors_from_mealy(mealy) - free_edges, _reference, _source = _free_parameterization(joint, len(idx)) - return [(idx.state(i), symbol, idx.state(j)) for i, symbol, j in free_edges] - - -def standard_errors( - hmm: HiddenMarkovModel, - observations: Sequence[Any], -) -> dict[tuple[Hashable, Any, Hashable], float]: - r"""Return asymptotic standard errors of the free edge parameters. - - Standard errors are ``sqrt(diag(J^{-1}))`` where ``J`` is the - :func:`observed_information` matrix (Cappe, Moulines & Ryden, 2005, - Section 10.2.3). Uses the Moore-Penrose pseudoinverse when ``J`` is singular; - a non-positive variance estimate (numerically unidentified parameter) yields - ``nan``. Keyed by the labels from :func:`free_parameter_labels`. - """ - labels = free_parameter_labels(hmm) - information = observed_information(hmm, observations) - if information.shape[0] == 0: - return {} - try: - covariance = np.linalg.inv(information) - except np.linalg.LinAlgError: - covariance = np.linalg.pinv(information) - variances = np.diag(covariance) - with np.errstate(invalid="ignore"): - errors = np.where(variances > 0.0, np.sqrt(variances), np.nan) - return dict(zip(labels, (float(value) for value in errors), strict=True)) - - -def _log_probabilities(values: np.ndarray) -> np.ndarray: - log_values = np.full(values.shape, -np.inf, dtype=float) - positive = values > 0.0 - log_values[positive] = np.log(values[positive]) - return log_values - - -def viterbi(hmm: HiddenMarkovModel, observations: Sequence[Any]) -> list[Hashable]: - mealy = _as_mealy_hmm(hmm) - idx = mealy.reindex() - pi, joint = _emission_transition_tensors_from_mealy(mealy) - n = len(idx) - obs = list(observations) - if n == 0: - return [] - if not obs: - if not np.any(pi > 0.0): - return [] - return [idx.state(int(np.argmax(pi)))] - - log_pi = _log_probabilities(pi) - viterbi_log = np.full((len(obs), n), -np.inf, dtype=float) - backpointer = np.full((len(obs), n), -1, dtype=int) - - matrix0 = joint.get(obs[0]) - if matrix0 is not None: - log_matrix0 = _log_probabilities(matrix0) - for j in range(n): - best = log_pi + log_matrix0[:, j] - viterbi_log[0, j] = np.max(best) - backpointer[0, j] = int(np.argmax(best)) - - for t in range(1, len(obs)): - matrix = joint.get(obs[t]) - if matrix is None: - continue - log_matrix = _log_probabilities(matrix) - for j in range(n): - scores = viterbi_log[t - 1] + log_matrix[:, j] - viterbi_log[t, j] = np.max(scores) - backpointer[t, j] = int(np.argmax(scores)) - - if not np.any(np.isfinite(viterbi_log[-1])): - return [] - - path = [0] * len(obs) - path[-1] = int(np.argmax(viterbi_log[-1])) - for t in range(len(obs) - 2, -1, -1): - path[t] = backpointer[t + 1, path[t + 1]] - return [idx.state(i) for i in path] - - -def sample( - hmm: HiddenMarkovModel, - n: int, - rng: np.random.Generator | None = None, -) -> tuple[list[Any], list[Hashable]]: - generator = rng if rng is not None else np.random.default_rng() - mealy = _as_mealy_hmm(hmm) - idx = mealy.reindex() - pi, joint = _emission_transition_tensors_from_mealy(mealy) - state = int(generator.choice(len(idx), p=pi / pi.sum())) - - observations: list[Any] = [] - states: list[Hashable] = [] - for _ in range(n): - states.append(idx.state(state)) - row_sum = sum(matrix[state].sum() for matrix in joint.values()) - if row_sum <= 0.0: - break - symbol_probs = np.array([joint[sym][state].sum() for sym in joint], dtype=float) - symbol_probs /= symbol_probs.sum() - symbol_index = int(generator.choice(len(joint), p=symbol_probs)) - symbol = list(joint.keys())[symbol_index] - observations.append(symbol) - matrix = joint[symbol] - row = matrix[state] - if row.sum() <= 0.0: - break - state = int(generator.choice(len(idx), p=row / row.sum())) - return observations, states diff --git a/sofic/generators/markov.py b/sofic/generators/markov.py index e2d61dd..60a2651 100644 --- a/sofic/generators/markov.py +++ b/sofic/generators/markov.py @@ -77,15 +77,28 @@ def entropy_rate(self) -> float: def words_of_length(self, length: int) -> dict[tuple[Hashable, ...], float]: """Return visible state paths of ``length`` and their probabilities.""" - from sofic.generators.words import markov_words_of_length + from sofic.generators.words import _markov_words_of_length - return markov_words_of_length(self, length) + return _markov_words_of_length(self, length) def sample_path(self, n: int, rng: np.random.Generator | None = None) -> list[Hashable]: + """Sample a state path of length ``n``. + + Starts from :attr:`initial_distribution`, or from the stationary + distribution when no initial distribution is given. + """ generator = rng if rng is not None else np.random.default_rng() idx = self.reindex() - pi = self.stationary_distribution() - state = int(generator.choice(len(idx), p=pi)) + if self.initial_distribution: + pi = np.zeros(len(idx), dtype=float) + for state, mass in self.initial_distribution.items(): + pi[idx.index(state)] = float(mass) + else: + pi = np.asarray(self.stationary_distribution(), dtype=float) + total = float(pi.sum()) + if not total > 0.0: + raise ValueError("cannot sample: the initial state distribution has no mass") + state = int(generator.choice(len(idx), p=pi / total)) path: list[Hashable] = [] for _ in range(n): diff --git a/sofic/generators/matrices.py b/sofic/generators/matrices.py new file mode 100644 index 0000000..f1bca36 --- /dev/null +++ b/sofic/generators/matrices.py @@ -0,0 +1,252 @@ +"""Symbol-labeled transition matrices and start vectors for stochastic generators. + +For a Mealy-style hidden Markov model with states indexed by ``model.reindex()``, +the symbol-labeled joint transition matrices are + +.. math:: T^{(x)}_{ij} = P(S_{t+1} = j, X_t = x \\mid S_t = i), + +so the probability of a word ``w = x_0 ... x_{L-1}`` from a start vector +:math:`\\eta` is :math:`\\eta T^{(x_0)} \\cdots T^{(x_{L-1})} \\mathbf{1}` +(:cite:`Rabiner1989,Ellison2009`). Their sum +:math:`T = \\sum_x T^{(x)}` is the internal state-to-state transition matrix. + +Start policies +-------------- +Every helper that needs an initial state law takes ``policy``: + +``"model"`` + The model's ``initial_distribution``, or its ``stationary_distribution()`` + when no initial distribution is given. This is the law of the generator *as + specified* and is used for likelihoods, decoding, sampling, and finite-word + probabilities. + +``"stationary"`` + The stationary law of the process. Block and window statistics of a + stationary process must weight the first state by this law rather than by a + possibly transient ``initial_distribution``. On reducible chains the + eigenvector stationary law is not unique, so the limit of the ``"model"`` + start vector under :math:`T` is preferred; the eigenvector solution is used + when that limit cannot be formed (or the model is symbolic), and the + ``"model"`` start vector as a last resort. + +An explicit ``start`` (a state, a state-to-mass mapping, or a dense vector in +``reindex()`` order) overrides either policy. +""" + +from __future__ import annotations + +from collections.abc import Callable, Hashable, Iterable, Mapping, Sequence +from typing import Any, Literal + +import numpy as np + +from sofic.generators.prob import as_prob, has_symbolic, zeros +from sofic.graph import ATTR_EMISSION, ATTR_PROB + +StartPolicy = Literal["model", "stationary"] +StartSpec = Hashable | Mapping[Hashable, Any] | Sequence[Any] | np.ndarray | None + +SKIP = object() + + +def accumulate_matrices( + model: Any, + *, + attr: str = ATTR_PROB, + states: Iterable[Hashable] | None = None, + label: Callable[[Any], Any] = lambda transition: None, + labels: Iterable[Any] = (), + symbolic: bool | None = None, +) -> tuple[dict[Any, np.ndarray], list[Hashable]]: + """Accumulate edge ``attr`` weights into one state-to-state matrix per edge label. + + Rows and columns follow ``states`` when given (edges leaving the set are + ignored), otherwise all model states in iteration order. ``label(transition)`` + keys the matrix an edge contributes to; returning ``SKIP`` drops the edge. + Matrices for ``labels`` are allocated first, in the given order, so they exist + even without edges; other labels follow in order of first appearance. + ``symbolic`` selects object (sympy) storage and is inferred from the retained + edge weights when omitted. Returns the matrices and the ordered state list. + """ + ordered = list(states) if states is not None else list(model.states()) + index = {state: i for i, state in enumerate(ordered)} + n = len(ordered) + edges: list[tuple[int, int, Any, Any]] = [] + for state in ordered: + for transition in model.graph.out_transitions(state): + j = index.get(transition.target) + key = label(transition) + if j is None or key is SKIP: + continue + edges.append((index[state], j, key, transition.data.get(attr, 0.0))) + if symbolic is None: + symbolic = has_symbolic(value for *_, value in edges) + + matrices: dict[Any, np.ndarray] = {key: zeros((n, n), symbolic=symbolic) for key in labels} + for i, j, key, value in edges: + matrix = matrices.get(key) + if matrix is None: + matrix = matrices[key] = zeros((n, n), symbolic=symbolic) + if symbolic: + matrix[i, j] = as_prob(matrix[i, j]) + as_prob(value) + else: + matrix[i, j] += float(value) + return matrices, ordered + + +def emission_label(transition: Any) -> Any: + """:func:`accumulate_matrices` label keying an edge by its emitted symbol (unlabeled edges skipped).""" + emission = transition.data.get(ATTR_EMISSION) + return SKIP if emission is None else emission + + +def _as_mealy(model: Any) -> Any: + to_mealy = getattr(model, "to_mealy", None) + return to_mealy() if to_mealy is not None else model + + +def _is_symbolic(mealy: Any) -> bool: + edge_probs = [transition.data.get(ATTR_PROB, 0.0) for transition in mealy.transitions()] + return has_symbolic(edge_probs) or has_symbolic(mealy.initial_distribution.values()) + + +def symbol_matrices(mealy: Any) -> dict[Any, np.ndarray]: + """Return symbol -> joint transition matrix ``T^(x)`` of a Mealy-style model. + + Symbols (the observation alphabet plus every emitted symbol) are keyed in + ``repr``-sorted order, so iteration and seeded sampling are reproducible. + Matrices are object-dtype when any edge or initial probability is symbolic. + """ + emissions = {transition.data.get(ATTR_EMISSION) for transition in mealy.transitions()} - {None} + symbols = sorted(set(getattr(mealy, "observation_alphabet", ())) | emissions, key=repr) + matrices, _states = accumulate_matrices( + mealy, + attr=ATTR_PROB, + states=mealy.reindex().states, + label=emission_label, + labels=symbols, + symbolic=_is_symbolic(mealy), + ) + return matrices + + +def limit_distribution(pi_initial: np.ndarray, transition: np.ndarray) -> np.ndarray | None: + """Return the limiting occupation law of ``pi_initial`` under ``transition``. + + Power-iterates the normalized law until it is invariant. Returns ``None`` when + the mass vanishes; on periodic chains the result after the iteration cap need + not be invariant, so callers should check it. + """ + pi = np.asarray(pi_initial, dtype=float).copy() + total = float(pi.sum()) + if total <= 0.0: + return None + pi /= total + matrix = np.asarray(transition, dtype=float) + n = len(pi) + for _ in range(max(100, 20 * n)): + nxt = pi @ matrix + mass = float(nxt.sum()) + if mass <= 0.0: + return None + nxt /= mass + if np.allclose(nxt, pi, rtol=1e-12, atol=1e-14): + pi = nxt + break + pi = nxt + pi[np.isclose(pi, 0.0, atol=1e-15)] = 0.0 + mass = float(pi.sum()) + if mass <= 0.0: + return None + return pi / mass + + +def _model_start(mealy: Any) -> np.ndarray: + idx = mealy.reindex() + n = len(idx) + symbolic = _is_symbolic(mealy) + if mealy.initial_distribution: + pi = zeros((n,), symbolic=symbolic) + for state, mass in mealy.initial_distribution.items(): + pi[idx.index(state)] = as_prob(mass) + return pi + if n: + return np.asarray(mealy.stationary_distribution(), dtype=object if symbolic else float) + return zeros((0,), symbolic=symbolic) + + +def _stationary_start(mealy: Any, joint: Mapping[Any, np.ndarray]) -> np.ndarray: + from sofic.generators.stationary import stationary_distribution_from_transition + + pi_initial = _model_start(mealy) + n = len(pi_initial) + if n == 0: + return pi_initial + symbolic = pi_initial.dtype == object or any(matrix.dtype == object for matrix in joint.values()) + transition = zeros((n, n), symbolic=symbolic) + for matrix in joint.values(): + transition = transition + matrix + if not symbolic: + limited = limit_distribution(pi_initial, transition) + if limited is not None and np.allclose(limited @ transition, limited, rtol=1e-8, atol=1e-10): + return limited + try: + return stationary_distribution_from_transition(transition) + except (np.linalg.LinAlgError, ValueError): + return pi_initial + + +def _explicit_start(mealy: Any, start: Hashable | Mapping[Hashable, Any] | Sequence[Any] | np.ndarray) -> np.ndarray: + idx = mealy.reindex() + n = len(idx) + if isinstance(start, Mapping): + vector = zeros((n,), symbolic=has_symbolic(start.values())) + for state, mass in start.items(): + if not mealy.graph.has_state(state): + raise ValueError(f"unknown start state {state!r}") + vector[idx.index(state)] = as_prob(mass) + return vector + + if isinstance(start, Hashable) and mealy.graph.has_state(start): + vector = zeros((n,)) + vector[idx.index(start)] = 1.0 + return vector + + values = list(np.asarray(start, dtype=object).ravel()) + vector = np.asarray(values, dtype=object if has_symbolic(values) else float) + if vector.shape != (n,): + raise ValueError(f"start vector has length {vector.size}, expected {n}") + return vector + + +def start_vector(model: Any, start: StartSpec = None, *, policy: StartPolicy = "model") -> np.ndarray: + """Return the initial state vector of ``model`` in ``model.to_mealy().reindex()`` order. + + ``start`` may be ``None`` (use ``policy``; see the module docstring), a state + (point mass), a state -> mass mapping, or a dense vector in state order. + Explicit starts are not renormalized. The vector is object-dtype when any + supplied or model probability is symbolic. + """ + mealy = _as_mealy(model) + if start is not None: + return _explicit_start(mealy, start) + if policy == "model": + return _model_start(mealy) + if policy == "stationary": + return _stationary_start(mealy, symbol_matrices(mealy)) + raise ValueError(f"unknown start policy {policy!r}; expected 'model' or 'stationary'") + + +def emission_tensors(model: Any, *, policy: StartPolicy = "model") -> tuple[np.ndarray, dict[Any, np.ndarray]]: + """Return ``(pi, T)``: the ``policy`` start vector and symbol -> ``T^(x)`` matrices. + + ``model`` is converted with ``to_mealy()`` when it provides one, and both + outputs share that presentation's ``reindex()`` state order. + """ + mealy = _as_mealy(model) + joint = symbol_matrices(mealy) + if policy == "model": + return _model_start(mealy), joint + if policy == "stationary": + return _stationary_start(mealy, joint), joint + raise ValueError(f"unknown start policy {policy!r}; expected 'model' or 'stationary'") diff --git a/sofic/generators/measures.py b/sofic/generators/measures.py index 62bb5f0..0bf1013 100644 --- a/sofic/generators/measures.py +++ b/sofic/generators/measures.py @@ -77,31 +77,27 @@ def state_entropy(model: StochasticModel) -> Any: def joint_block_distribution( generator: HiddenMarkovModel, - history_length: int = 1, + block_length: int = 2, ) -> Any: - """Build a ``dit.Distribution`` over observed emission blocks. - - ``history_length`` counts symbols before the present symbol, so the emitted - block length is ``history_length + 1``. - """ - from itertools import product - - from sofic.generators.hmm_inference import _stationary_emission_tensors + """Build a ``dit.Distribution`` over observed emission blocks of ``block_length`` symbols.""" + from sofic.generators.matrices import emission_tensors + from sofic.generators.words import _enumerate_words, _matrix_step + if block_length < 1: + raise ValueError("block_length must be at least 1") dit = _require_dit() # Blocks of a stationary process are weighted by the stationary state law, not # the model's initial distribution (which may describe only the transient). - pi, joint = _stationary_emission_tensors(generator) + pi, joint = emission_tensors(generator, policy="stationary") symbol_list = sorted(generator.observation_alphabet, key=repr) - block_length = max(1, history_length + 1) ones = np.ones(len(pi), dtype=float) - outcomes = list(product(symbol_list, repeat=block_length)) + outcomes = [] probs = [] - for outcome in outcomes: - mass = pi.copy() - for symbol in outcome: - mass = mass @ joint.get(symbol, np.zeros((len(pi), len(pi)), dtype=float)) + for outcome, mass in _enumerate_words( + symbol_list, block_length, pi.copy(), _matrix_step(joint, len(pi), prune=False) + ): + outcomes.append(outcome) probs.append(float(mass @ ones)) total = sum(probs) @@ -168,7 +164,7 @@ def entropy_rate_hmm(hmm: HiddenMarkovModel) -> Any: Returns a sympy :class:`~sympy.Expr` when the stationary law or emission tensors are symbolic; otherwise a Python ``float``. """ - from sofic.generators.hmm_inference import _emission_transition_tensors + from sofic.generators.matrices import symbol_matrices from sofic.generators.prob import ( array_sum, as_prob, @@ -186,7 +182,7 @@ def entropy_rate_hmm(hmm: HiddenMarkovModel) -> Any: dit = _require_dit() idx = hmm.reindex() pi = hmm.stationary_distribution() - _, joint = _emission_transition_tensors(hmm) + joint = symbol_matrices(hmm.to_mealy()) symbolic = pi.dtype == object or has_symbolic(pi.ravel()) if not symbolic: @@ -239,15 +235,15 @@ def entropy_rate_markov(chain: MarkovChain) -> Any: def collision_entropy(quasi_model: QuasiStochasticModel) -> float: - """Second Renyi entropy rate from quasi transition matrices.""" - matrices = quasi_model.transition_matrices() + """Second Renyi entropy rate (bits) from quasi transition matrices.""" + matrices = quasi_model.symbol_matrices() pi = quasi_model.stationary_quasidistribution() total = 0.0 for matrix in matrices.values(): total += float(pi @ (matrix @ matrix) @ np.ones(len(pi))) if total <= 0.0: return 0.0 - return float(-np.log(total)) + return float(-np.log2(total)) def process_negativity(quasi_model: QuasiStochasticModel) -> float: diff --git a/sofic/generators/mixed_state_construction.py b/sofic/generators/mixed_state_construction.py index 4968fc0..840bdd0 100644 --- a/sofic/generators/mixed_state_construction.py +++ b/sofic/generators/mixed_state_construction.py @@ -10,6 +10,7 @@ from sofic.exceptions import MixedStateExplosionError from sofic.generators.base import HiddenMarkovModel +from sofic.generators.matrices import start_vector, symbol_matrices from sofic.generators.mealy import MealyHMM from sofic.generators.mixed_state import ( MixedState, @@ -18,12 +19,10 @@ ) from sofic.generators.prob import ( as_prob, - has_symbolic, is_positive_mass, matvec, simplify_prob, sum_probs, - zeros, ) from sofic.graph import ATTR_EMISSION, ATTR_PROB, TransitionGraph @@ -41,14 +40,8 @@ def _resolve_initial_belief( vector = hmm.stationary_distribution() elif isinstance(initial_mixed_state, MixedState): vector = initial_mixed_state.as_array() - elif isinstance(initial_mixed_state, Mapping): - index = {state: i for i, state in enumerate(basis)} - symbolic = has_symbolic(initial_mixed_state.values()) - vector = zeros((len(basis),), symbolic=symbolic) - for state, mass in initial_mixed_state.items(): - vector[index[state]] = as_prob(mass) else: - vector = np.asarray(initial_mixed_state, dtype=object if has_symbolic(initial_mixed_state) else float) + vector = start_vector(hmm, initial_mixed_state) if len(np.asarray(vector).ravel()) != len(basis): raise ValueError(f"initial belief has length {len(np.asarray(vector).ravel())}, expected {len(basis)}") @@ -84,13 +77,11 @@ def build_mixed_state_presentation( if not isinstance(hmm, MealyHMM): raise TypeError(f"mixed-state presentation requires a MealyHMM, not {type(hmm)!r}") - from sofic.generators.hmm_inference import _emission_transition_tensors - constraints = getattr(hmm, "symbol_constraints", None) idx = hmm.reindex() basis = idx.states - _, joint = _emission_transition_tensors(hmm) + joint = symbol_matrices(hmm) symbols = tuple(sorted(joint, key=str)) eta0 = _resolve_initial_belief(hmm, basis, initial_mixed_state) diff --git a/sofic/generators/pfa.py b/sofic/generators/pfa.py index 71c9e2c..dd7251d 100644 --- a/sofic/generators/pfa.py +++ b/sofic/generators/pfa.py @@ -72,9 +72,9 @@ def string_probability(self, word: Sequence[Any]) -> float: def words_of_length(self, length: int) -> dict[tuple[Any, ...], float]: """Return output words of ``length`` and their probabilities.""" - from sofic.generators.words import pfa_words_of_length + from sofic.generators.words import _pfa_words_of_length - return pfa_words_of_length(self, length) + return _pfa_words_of_length(self, length) def sample(self, n: int, rng: np.random.Generator | None = None) -> list[Any]: generator = rng if rng is not None else np.random.default_rng() diff --git a/sofic/generators/process_equivalence.py b/sofic/generators/process_equivalence.py index f1c22d8..bd5881f 100644 --- a/sofic/generators/process_equivalence.py +++ b/sofic/generators/process_equivalence.py @@ -14,8 +14,7 @@ import numpy as np from sofic.generators.base import HiddenMarkovModel -from sofic.generators.hmm_inference import _emission_transition_tensors_from_mealy -from sofic.generators.words import _start_vector +from sofic.generators.matrices import start_vector, symbol_matrices _DEFAULT_RTOL = 1e-9 _DEFAULT_ATOL = 1e-12 @@ -77,11 +76,10 @@ def from_hmm( start: Hashable | Mapping[Hashable, float] | Sequence[float] | np.ndarray | None = None, ) -> _HistoryFutureWordList: mealy = hmm.to_mealy() - pi, matrices = _emission_transition_tensors_from_mealy(mealy) return cls( alphabet=tuple(sorted(mealy.observation_alphabet, key=repr)), - matrices=matrices, - start=_start_vector(mealy, pi, start), + matrices=symbol_matrices(mealy), + start=np.asarray(start_vector(mealy, start), dtype=float), ) @property diff --git a/sofic/generators/quasi_inference.py b/sofic/generators/quasi_inference.py index 2a467d2..8302a5a 100644 --- a/sofic/generators/quasi_inference.py +++ b/sofic/generators/quasi_inference.py @@ -9,21 +9,14 @@ from sofic.exceptions import QuasiStochasticValidationError from sofic.generators.base import QuasiStochasticModel -from sofic.graph import ATTR_EMISSION, ATTR_QUASIPROB +from sofic.generators.matrices import accumulate_matrices, emission_label +from sofic.graph import ATTR_QUASIPROB -def transition_matrices(model: QuasiStochasticModel) -> dict[Any, np.ndarray]: - idx = model.reindex() - n = len(idx) - matrices: dict[Any, np.ndarray] = {} - for transition in model.transitions(): - emission = transition.data.get(ATTR_EMISSION) - if emission is None: - continue - matrix = matrices.setdefault(emission, np.zeros((n, n), dtype=float)) - i = idx.index(transition.source) - j = idx.index(transition.target) - matrix[i, j] += float(transition.data.get(ATTR_QUASIPROB, 0.0)) +def symbol_matrices(model: QuasiStochasticModel) -> dict[Any, np.ndarray]: + matrices, _states = accumulate_matrices( + model, attr=ATTR_QUASIPROB, states=model.reindex().states, label=emission_label, symbolic=False + ) return matrices @@ -34,7 +27,7 @@ def stationary_quasidistribution(model: QuasiStochasticModel) -> np.ndarray: return np.array([], dtype=float) combined = np.zeros((n, n), dtype=float) - for matrix in transition_matrices(model).values(): + for matrix in symbol_matrices(model).values(): combined += matrix distribution = np.zeros(n, dtype=float) @@ -64,7 +57,7 @@ def word_probability(model: QuasiStochasticModel, word: Sequence[Any]) -> float: for state, mass in model.initial_quasidistribution.items(): pi[idx.index(state)] = float(mass) ones = np.ones(n, dtype=float) - matrices = transition_matrices(model) + matrices = symbol_matrices(model) result = pi for symbol in word: matrix = matrices.get(symbol) diff --git a/sofic/generators/quasi_realization.py b/sofic/generators/quasi_realization.py index c7825fd..b52eaa9 100644 --- a/sofic/generators/quasi_realization.py +++ b/sofic/generators/quasi_realization.py @@ -34,7 +34,7 @@ def validate_quasistochastic(self) -> None: if not np.isclose(self.pi.sum(), 1.0): raise QuasiStochasticValidationError(f"pi sums to {self.pi.sum()}, not 1") - def transition_matrices(self) -> dict[Any, np.ndarray]: + def symbol_matrices(self) -> dict[Any, np.ndarray]: return dict(self.symbol_maps) def stationary_quasidistribution(self) -> np.ndarray: diff --git a/sofic/generators/sampling.py b/sofic/generators/sampling.py new file mode 100644 index 0000000..d11a90a --- /dev/null +++ b/sofic/generators/sampling.py @@ -0,0 +1,45 @@ +"""Sampling observation and hidden-state sequences from hidden Markov models.""" + +from __future__ import annotations + +from collections.abc import Hashable +from typing import Any + +import numpy as np + +from sofic.generators.base import HiddenMarkovModel +from sofic.generators.matrices import emission_tensors + + +def sample( + hmm: HiddenMarkovModel, + n: int, + rng: np.random.Generator | None = None, +) -> tuple[list[Any], list[Hashable]]: + generator = rng if rng is not None else np.random.default_rng() + mealy = hmm.to_mealy() + idx = mealy.reindex() + pi, joint = emission_tensors(mealy) + total = float(np.sum(pi)) + if not total > 0.0: + raise ValueError("cannot sample: the initial state distribution has no mass") + state = int(generator.choice(len(idx), p=pi / total)) + + observations: list[Any] = [] + states: list[Hashable] = [] + for _ in range(n): + states.append(idx.state(state)) + row_sum = sum(matrix[state].sum() for matrix in joint.values()) + if row_sum <= 0.0: + break + symbol_probs = np.array([joint[sym][state].sum() for sym in joint], dtype=float) + symbol_probs /= symbol_probs.sum() + symbol_index = int(generator.choice(len(joint), p=symbol_probs)) + symbol = list(joint.keys())[symbol_index] + observations.append(symbol) + matrix = joint[symbol] + row = matrix[state] + if row.sum() <= 0.0: + break + state = int(generator.choice(len(idx), p=row / row.sum())) + return observations, states diff --git a/sofic/generators/topological_epsilon_enumeration.py b/sofic/generators/topological_epsilon_enumeration.py index 4e24939..80c4d9b 100644 --- a/sofic/generators/topological_epsilon_enumeration.py +++ b/sofic/generators/topological_epsilon_enumeration.py @@ -11,15 +11,15 @@ import numpy as np -from sofic.automata.idfa import ( +from sofic.automata.enumeration.idfa import ( MISSING_TRANSITION, IDFAEnumerationError, _delta_table, idfa_string_to_topological_graph, + idfa_transition_count, iter_idfa_strings, rank_idfa_string, reroot_idfa_string, - transition_count, validate_idfa_string, ) from sofic.exceptions import SoficValidationError @@ -128,7 +128,7 @@ def epsilon_machine_to_idfa_string( """Encode an ε-machine as an incomplete accessible DFA transition string. Probabilities are ignored. Missing symbol transitions are encoded with - :data:`sofic.automata.idfa.MISSING_TRANSITION`. If ``canonical`` is true, + :data:`sofic.automata.enumeration.idfa.MISSING_TRANSITION`. If ``canonical`` is true, all states are tried as roots and the rank-minimal IDFA string is returned. Otherwise, the first state in deterministic label order is used as the root. """ @@ -274,7 +274,7 @@ def is_topological_epsilon_string( ) -> bool: """Return whether ``transitions`` passes the structural ε-machine tests.""" validate_idfa_string(transitions, n=n, k=k) - defined = transition_count(transitions) + defined = idfa_transition_count(transitions) if defined < n: return False if n > 1 and defined >= n * k: diff --git a/sofic/generators/words.py b/sofic/generators/words.py index 171a999..904471a 100644 --- a/sofic/generators/words.py +++ b/sofic/generators/words.py @@ -2,26 +2,72 @@ from __future__ import annotations -from collections.abc import Hashable, Mapping, Sequence -from itertools import product +from collections.abc import Callable, Hashable, Iterator, Mapping, Sequence from typing import Any import numpy as np from sofic.generators.base import HiddenMarkovModel, QuasiStochasticModel -from sofic.generators.hmm_inference import _emission_transition_tensors from sofic.generators.markov import MarkovChain +from sofic.generators.matrices import emission_tensors, start_vector, symbol_matrices from sofic.generators.pfa import ProbabilisticFiniteAutomaton from sofic.graph import ATTR_PROB _TOL = 1e-15 -def hmm_words_of_length(hmm: HiddenMarkovModel, length: int) -> dict[tuple[Any, ...], float]: +def _enumerate_words( + alphabet: Sequence[Any], + length: int, + start: Any = None, + step: Callable[[Any, Any], Any] | None = None, +) -> Iterator[tuple[tuple[Any, ...], Any]]: + """Yield ``(word, mass)`` for every word of ``length`` over ``alphabet``. + + Words come in lexicographic (``itertools.product``) order of ``alphabet``. + ``mass`` starts at ``start`` and is advanced by ``step(mass, symbol)`` once per + symbol, so prefixes are propagated once rather than per word; a step returning + ``None`` prunes every extension of that prefix. With ``step=None`` the mass is + carried unchanged and every word is yielded. + """ + + def walk(prefix: tuple[Any, ...], mass: Any, remaining: int) -> Iterator[tuple[tuple[Any, ...], Any]]: + if remaining == 0: + yield prefix, mass + return + for symbol in alphabet: + if step is None: + yield from walk(prefix + (symbol,), mass, remaining - 1) + continue + nxt = step(mass, symbol) + if nxt is not None: + yield from walk(prefix + (symbol,), nxt, remaining - 1) + + yield from walk((), start, length) + + +def _matrix_step(matrices: Mapping[Any, np.ndarray], n: int, *, prune: bool = True) -> Callable[[np.ndarray, Any], Any]: + """Return a :func:`_enumerate_words` step multiplying a row mass by ``matrices[symbol]``. + + Symbols without a matrix act as the zero matrix. With ``prune`` a numeric + prefix whose mass vector is identically zero is dropped. + """ + zero = np.zeros((n, n), dtype=float) + + def step(mass: np.ndarray, symbol: Any) -> np.ndarray | None: + nxt = mass @ matrices.get(symbol, zero) + if prune and nxt.dtype != object and not np.any(nxt): + return None + return nxt + + return step + + +def _hmm_words_of_length(hmm: HiddenMarkovModel, length: int) -> dict[tuple[Any, ...], float]: """Return observed words of ``length`` and their probabilities.""" if length < 0: raise ValueError("length must be nonnegative") - pi, joint = _emission_transition_tensors(hmm) + pi, joint = emission_tensors(hmm) alphabet = sorted(hmm.observation_alphabet, key=repr) if length == 0: return {(): float(pi.sum())} if pi.sum() > _TOL else {} @@ -29,19 +75,15 @@ def hmm_words_of_length(hmm: HiddenMarkovModel, length: int) -> dict[tuple[Any, return {} terminal = np.ones(len(pi), dtype=float) - zero = np.zeros((len(pi), len(pi)), dtype=float) distribution: dict[tuple[Any, ...], float] = {} - for word in product(alphabet, repeat=length): - mass = pi.copy() - for symbol in word: - mass = mass @ joint.get(symbol, zero) + for word, mass in _enumerate_words(alphabet, length, pi.copy(), _matrix_step(joint, len(pi))): probability = float(mass @ terminal) if abs(probability) > _TOL: distribution[word] = probability return distribution -def hmm_word_probability( +def _hmm_word_probability( hmm: HiddenMarkovModel, word: Sequence[Any], *, @@ -53,8 +95,8 @@ def hmm_word_probability( a state-probability mapping, or a dense vector in the model's state order. """ mealy = hmm.to_mealy() - pi, joint = _emission_transition_tensors(mealy) - mass = _start_vector(mealy, pi, start) + joint = symbol_matrices(mealy) + mass = np.asarray(start_vector(mealy, start), dtype=float) if len(word) == 0: return float(mass.sum()) n = len(mass) @@ -67,20 +109,20 @@ def hmm_word_probability( return float(mass.sum()) -def hmm_log_word_probability( +def _hmm_log_word_probability( hmm: HiddenMarkovModel, word: Sequence[Any], *, start: Hashable | Mapping[Hashable, float] | Sequence[float] | np.ndarray | None = None, ) -> float: """Return ``log2(P(word))`` or ``-inf`` for forbidden words.""" - probability = hmm_word_probability(hmm, word, start=start) + probability = _hmm_word_probability(hmm, word, start=start) if probability <= 0.0: return float("-inf") return float(np.log2(probability)) -def hmm_word_probabilities( +def _hmm_word_probabilities( hmm: HiddenMarkovModel, lengths: int | Sequence[int], *, @@ -97,18 +139,18 @@ def hmm_word_probabilities( distribution: dict[tuple[Any, ...], float] = {} for length in requested: if length == 0: - probability = hmm_word_probability(mealy, (), start=start) + probability = _hmm_word_probability(mealy, (), start=start) if not sparse or abs(probability) > _TOL: distribution[()] = probability continue - for word in product(alphabet, repeat=length): - probability = hmm_word_probability(mealy, word, start=start) + for word, _ in _enumerate_words(alphabet, length): + probability = _hmm_word_probability(mealy, word, start=start) if not sparse or abs(probability) > _TOL: distribution[word] = probability return distribution -def hmm_conditional_word_probability( +def _hmm_conditional_word_probability( hmm: HiddenMarkovModel, word: Sequence[Any], condition: Sequence[Any], @@ -116,14 +158,14 @@ def hmm_conditional_word_probability( start: Hashable | Mapping[Hashable, float] | Sequence[float] | np.ndarray | None = None, ) -> float: """Return ``P(word | condition)`` from the requested start distribution.""" - condition_probability = hmm_word_probability(hmm, condition, start=start) + condition_probability = _hmm_word_probability(hmm, condition, start=start) if condition_probability <= _TOL: raise ZeroDivisionError("condition has zero probability") joint_word = tuple(condition) + tuple(word) - return hmm_word_probability(hmm, joint_word, start=start) / condition_probability + return _hmm_word_probability(hmm, joint_word, start=start) / condition_probability -def pfa_words_of_length(pfa: ProbabilisticFiniteAutomaton, length: int) -> dict[tuple[Any, ...], float]: +def _pfa_words_of_length(pfa: ProbabilisticFiniteAutomaton, length: int) -> dict[tuple[Any, ...], float]: """Return output words of ``length`` and their probabilities.""" if length < 0: raise ValueError("length must be nonnegative") @@ -134,14 +176,14 @@ def pfa_words_of_length(pfa: ProbabilisticFiniteAutomaton, length: int) -> dict[ if not alphabet: return {} distribution: dict[tuple[Any, ...], float] = {} - for word in product(alphabet, repeat=length): + for word, _ in _enumerate_words(alphabet, length): probability = pfa.string_probability(word) if probability > _TOL: distribution[word] = probability return distribution -def quasi_words_of_length(model: QuasiStochasticModel, length: int) -> dict[tuple[Any, ...], float]: +def _quasi_words_of_length(model: QuasiStochasticModel, length: int) -> dict[tuple[Any, ...], float]: """Return words of ``length`` and their signed quasiprobabilities.""" if length < 0: raise ValueError("length must be nonnegative") @@ -152,23 +194,25 @@ def quasi_words_of_length(model: QuasiStochasticModel, length: int) -> dict[tupl if not alphabet: return {} distribution: dict[tuple[Any, ...], float] = {} - for word in product(sorted(alphabet, key=repr), repeat=length): + for word, _ in _enumerate_words(sorted(alphabet, key=repr), length): probability = float(model.word_probability(word)) if abs(probability) > _TOL: distribution[word] = probability return distribution -def markov_words_of_length(chain: MarkovChain, length: int) -> dict[tuple[Hashable, ...], float]: +def _markov_words_of_length(chain: MarkovChain, length: int) -> dict[tuple[Hashable, ...], float]: """Return visible state paths of ``length`` and their probabilities.""" if length < 0: raise ValueError("length must be nonnegative") states = tuple(chain.states()) + start = _markov_start(chain) if length == 0: - return {(): 1.0} + total = float(sum(start.values())) + return {(): total} if total > _TOL else {} distribution: dict[tuple[Hashable, ...], float] = {} - for word in product(states, repeat=length): - probability = _markov_path_probability(chain, word) + for word, _ in _enumerate_words(states, length): + probability = _markov_path_probability(chain, word, start) if probability > _TOL: distribution[word] = probability return distribution @@ -179,42 +223,22 @@ def _quasi_alphabet(model: QuasiStochasticModel) -> tuple[Any, ...]: alphabet = getattr(model, name, None) if alphabet: return tuple(alphabet) - return tuple(model.transition_matrices()) + return tuple(model.symbol_matrices()) -def _start_vector( - hmm: HiddenMarkovModel, - default: np.ndarray, - start: Hashable | Mapping[Hashable, float] | Sequence[float] | np.ndarray | None, -) -> np.ndarray: - if start is None: - return np.array(default, dtype=float) - - idx = hmm.reindex() - n = len(idx) - if isinstance(start, Mapping): - vector = np.zeros(n, dtype=float) - for state, mass in start.items(): - if not hmm.graph.has_state(state): - raise ValueError(f"unknown start state {state!r}") - vector[idx.index(state)] = float(mass) - return vector - - if hmm.graph.has_state(start): - vector = np.zeros(n, dtype=float) - vector[idx.index(start)] = 1.0 - return vector - - vector = np.asarray(start, dtype=float) - if vector.shape != (n,): - raise ValueError(f"start vector must have shape {(n,)}, got {vector.shape}") - return vector.copy() - - -def _markov_path_probability(chain: MarkovChain, path: tuple[Hashable, ...]) -> float: +def _markov_start(chain: MarkovChain) -> dict[Hashable, float]: + """Initial law of ``chain``, or its stationary law when none is given.""" + if chain.initial_distribution: + return {state: float(mass) for state, mass in chain.initial_distribution.items()} + idx = chain.reindex() + pi = chain.stationary_distribution() + return {idx.state(i): float(mass) for i, mass in enumerate(pi)} + + +def _markov_path_probability(chain: MarkovChain, path: tuple[Hashable, ...], start: Mapping[Hashable, float]) -> float: if not path: - return 1.0 - probability = float(chain.initial_distribution.get(path[0], 0.0)) + return float(sum(start.values())) + probability = float(start.get(path[0], 0.0)) for source, target in zip(path, path[1:], strict=False): edge_probability = 0.0 for transition in chain.graph.out_transitions(source): diff --git a/sofic/inference/__init__.py b/sofic/inference/__init__.py index 76fdf41..aa0d8d2 100644 --- a/sofic/inference/__init__.py +++ b/sofic/inference/__init__.py @@ -1,6 +1,23 @@ -"""Inference algorithms for stochastic generators.""" +"""Inference algorithms for stochastic generators. -from sofic.inference import bayesian +Learners are named ``learn__``; the subpackages and modules +(:mod:`~sofic.inference.bayesian`, :mod:`~sofic.inference.cssr`, +:mod:`~sofic.inference.diagnostics`, :mod:`~sofic.inference.hmm`, +:mod:`~sofic.inference.model_selection`, :mod:`~sofic.inference.spectral`) are +bound under their module names. +""" + +from sofic.inference import bayesian, cssr, diagnostics, hmm, model_selection, spectral +from sofic.inference.cssr import ( + learn_epsilon_machine_cssr, + learn_epsilon_machine_subtree, + learn_epsilon_transducer_cssr, + learn_stack_hmm_cssr, + learn_stack_hmm_mle, + learn_stack_hmm_papni, + learn_stack_hmm_subtree, + suggest_max_history, +) from sofic.inference.diagnostics import ( GoodnessOfFit, StructureStability, @@ -9,6 +26,19 @@ structure_stability, topology_key, ) +from sofic.inference.hmm import ( + backward, + baum_welch, + forward, + free_parameter_labels, + log_likelihood, + observed_information, + score, + smooth, + standard_errors, + two_slice_marginals, + viterbi, +) from sofic.inference.model_selection import ( ModelScores, WAICResult, @@ -24,6 +54,7 @@ from sofic.inference.spectral import ( SpectralInferenceError, hankel_matrices, + learn_epsilon_machine_spectral, learn_spectral_wfa, project_to_epsilon_machine, project_to_mealy, @@ -33,6 +64,12 @@ __all__ = [ "bayesian", + "cssr", + "diagnostics", + "hmm", + "model_selection", + "spectral", + "learn_epsilon_machine_cssr", "GoodnessOfFit", "StructureStability", "goodness_of_fit", @@ -55,5 +92,24 @@ "project_to_epsilon_machine", "project_to_mealy", "project_to_nmachine", + "learn_epsilon_machine_spectral", "spectral_singular_values", + "learn_epsilon_machine_subtree", + "suggest_max_history", + "learn_epsilon_transducer_cssr", + "learn_stack_hmm_cssr", + "learn_stack_hmm_subtree", + "learn_stack_hmm_mle", + "learn_stack_hmm_papni", + "baum_welch", + "viterbi", + "forward", + "backward", + "smooth", + "two_slice_marginals", + "log_likelihood", + "score", + "observed_information", + "standard_errors", + "free_parameter_labels", ] diff --git a/sofic/inference/bayesian/diversity.py b/sofic/inference/bayesian/diversity.py index d473c4d..73135e4 100644 --- a/sofic/inference/bayesian/diversity.py +++ b/sofic/inference/bayesian/diversity.py @@ -35,7 +35,6 @@ from sofic.generators.base import HiddenMarkovModel from sofic.generators.process_equivalence import _HistoryFutureWordList -from sofic.generators.words import hmm_words_of_length from sofic.inference.bayesian.counts import BayesianInferenceError from sofic.inference.bayesian.epsilon import EpsilonMachinePosterior @@ -177,7 +176,7 @@ def posterior_mean_word_distribution( machine = posterior.posterior_mean_machine(start_node) if machine is None: continue - words = hmm_words_of_length(machine, length) + words = machine.words_of_length(length) for word, prob in words.items(): distribution[word] = distribution.get(word, 0.0) + start_weight * float(prob) return distribution @@ -240,7 +239,7 @@ def posterior_process_diversity( distributions = [] for _ in range(n_samples): _start, machine = comparison.generate_sample(rng=generator) - distributions.append(hmm_words_of_length(machine, length)) + distributions.append(machine.words_of_length(length)) weights = [1.0 / n_samples] * n_samples process_div = _jsd_from_word_distributions(distributions, weights, alphabet, length) return PosteriorDiversityResult( diff --git a/sofic/inference/bayesian/hdp_hmm.py b/sofic/inference/bayesian/hdp_hmm.py index 300e722..64b3a6b 100644 --- a/sofic/inference/bayesian/hdp_hmm.py +++ b/sofic/inference/bayesian/hdp_hmm.py @@ -47,7 +47,7 @@ class HDPHMMPosterior: state_counts Number of occupied states in each retained draw (``len == len(samples)``). log_likelihoods - Data log-likelihood (natural log) of each retained draw. + Data log-likelihood (bits) of each retained draw. alphabet Sorted observation alphabet used by the sampler. """ @@ -110,7 +110,7 @@ def _ffbs( emit: np.ndarray, rng: np.random.Generator, ) -> tuple[np.ndarray, float]: - """Forward-filter backward-sample one sequence; return states and log-likelihood.""" + """Forward-filter backward-sample one sequence; return states and log-likelihood (bits).""" n_states = trans.shape[0] length = obs_idx.shape[0] alpha = np.empty((length, n_states)) @@ -122,7 +122,7 @@ def _ffbs( weights = emit[:, obs_idx[0]].copy() scale = weights.sum() alpha[0] = weights / scale - loglik += np.log(scale) + loglik += np.log2(scale) for t in range(1, length): predicted = alpha[t - 1] @ trans @@ -132,7 +132,7 @@ def _ffbs( weights = emit[:, obs_idx[t]].copy() scale = weights.sum() alpha[t] = weights / scale - loglik += np.log(scale) + loglik += np.log2(scale) states = np.empty(length, dtype=int) states[length - 1] = rng.choice(n_states, p=alpha[length - 1]) diff --git a/sofic/inference/cssr/__init__.py b/sofic/inference/cssr/__init__.py new file mode 100644 index 0000000..56f6284 --- /dev/null +++ b/sofic/inference/cssr/__init__.py @@ -0,0 +1,54 @@ +"""Causal-State Splitting Reconstruction and its relatives. + +Process CSSR follows Shalizi, Shalizi & Crutchfield (arXiv:cs/0210025); subtree +merging follows Crutchfield & Young (PRL 1989; PRE 1994); transCSSR generalizes +CSSR to ε-transducers (Barnett & Crutchfield, J. Stat. Phys. 161:2 (2015)); stack +CSSR lifts CSSR to (suffix, stack) configurations of hidden Markov stack models. +""" + +from sofic.inference.cssr.counts import ( + ConfigurationHistory, + History, + JointHistory, + JointSuffixCounts, + StackSuffixCounts, + SuffixCounts, +) +from sofic.inference.cssr.process import learn_epsilon_machine_cssr, suggest_max_history +from sofic.inference.cssr.significance import ( + MorphTest, + TableTest, + aggregates_differ, + morph_test_score, + morphs_differ, +) +from sofic.inference.cssr.stack import ( + learn_stack_hmm_cssr, + learn_stack_hmm_mle, + learn_stack_hmm_papni, + learn_stack_hmm_subtree, +) +from sofic.inference.cssr.subtree import learn_epsilon_machine_subtree +from sofic.inference.cssr.transducer import learn_epsilon_transducer_cssr + +__all__ = [ + "ConfigurationHistory", + "History", + "JointHistory", + "JointSuffixCounts", + "MorphTest", + "StackSuffixCounts", + "SuffixCounts", + "TableTest", + "aggregates_differ", + "learn_epsilon_machine_cssr", + "learn_stack_hmm_mle", + "learn_stack_hmm_papni", + "morph_test_score", + "morphs_differ", + "learn_stack_hmm_cssr", + "learn_stack_hmm_subtree", + "learn_epsilon_machine_subtree", + "suggest_max_history", + "learn_epsilon_transducer_cssr", +] diff --git a/sofic/inference/cssr/counts.py b/sofic/inference/cssr/counts.py new file mode 100644 index 0000000..bc3c523 --- /dev/null +++ b/sofic/inference/cssr/counts.py @@ -0,0 +1,249 @@ +"""Empirical history counts for the CSSR family. + +:class:`SuffixCounts` counts suffixes of a single process, :class:`JointSuffixCounts` +joint ``(input, output)`` pasts of a channel, and :class:`StackSuffixCounts` +``(suffix, stack)`` configurations of a visibly pushdown process. +""" + +from __future__ import annotations + +from collections import Counter, defaultdict +from collections.abc import Iterable, Iterator, Sequence +from dataclasses import dataclass, field +from typing import TYPE_CHECKING, Any, ClassVar + +if TYPE_CHECKING: + from sofic.automata.learning.papni import DyckAlphabet + +History = tuple[Any, ...] + +JointHistory = tuple[tuple[Any, Any], ...] + +ConfigurationHistory = tuple[tuple[Any, ...], tuple[Any, ...]] + + +def infer_alphabet(tokens: Sequence[Any], alphabet: Sequence[Any] | None) -> tuple[Any, ...]: + """``alphabet`` as a tuple, or the distinct ``tokens`` sorted by ``repr``.""" + return tuple(sorted(set(tokens), key=repr)) if alphabet is None else tuple(alphabet) + + +def iter_suffixes(tokens: tuple[Any, ...], max_length: int) -> Iterator[tuple[int, tuple[Any, ...]]]: + """Yield ``(t, tokens[t - L : t])`` for each position ``t`` and each ``L`` in ``0..min(t, max_length)``. + + These are the pasts, up to ``max_length`` long, that precede the token at ``t``. + """ + for t in range(len(tokens)): + for length in range(min(t, max_length) + 1): + yield t, tokens[t - length : t] + + +@dataclass +class SuffixCounts: + """Empirical counts of histories and following symbols in a sequence.""" + + alphabet: tuple[Any, ...] + history_counts: Counter[History] = field(default_factory=Counter) + next_counts: dict[History, Counter[Any]] = field(default_factory=lambda: defaultdict(Counter)) + + #: History key used as the fallback for an empty history set (overridden by stack counts). + empty_history: ClassVar[History] = () + + @classmethod + def from_sequence( + cls, + sequence: Sequence[Any], + *, + alphabet: Sequence[Any] | None = None, + max_length: int | None = None, + ) -> SuffixCounts: + seq = tuple(sequence) + if not seq: + raise ValueError("sequence must be non-empty") + alphabet = infer_alphabet(seq, alphabet) + unknown = set(seq) - set(alphabet) + if unknown: + raise ValueError(f"symbols {unknown!r} not in alphabet") + counts = cls(alphabet=alphabet) + for t, history in iter_suffixes(seq, max_length if max_length is not None else len(seq)): + counts.history_counts[history] += 1 + counts.next_counts[history][seq[t]] += 1 + return counts + + def morph(self, history: History, *, smoothing: float = 0.0) -> dict[Any, float]: + """MLE (optional additive smoothing) of P(next symbol | history).""" + counts = self.next_counts.get(history, Counter()) + total = sum(counts.values()) + if total == 0: + uniform = 1.0 / len(self.alphabet) + return dict.fromkeys(self.alphabet, uniform) + denom = total + smoothing * len(self.alphabet) + return {symbol: (counts.get(symbol, 0) + smoothing) / denom for symbol in self.alphabet} + + def state_morph(self, histories: set[History], *, smoothing: float = 0.0) -> dict[Any, float]: + """Weighted average of history morphs with weights from occurrence counts.""" + weights = {history: float(self.history_counts.get(history, 0)) for history in histories} + total_weight = sum(weights.values()) + if total_weight <= 0.0: + return self.morph(self.empty_history, smoothing=smoothing) + result = dict.fromkeys(self.alphabet, 0.0) + for history, weight in weights.items(): + morph = self.morph(history, smoothing=smoothing) + for symbol in self.alphabet: + result[symbol] += weight * morph[symbol] + return {symbol: prob / total_weight for symbol, prob in result.items()} + + def marginal_morph(self) -> dict[Any, float]: + """Global next-symbol distribution (IID morph at L=0).""" + counts = Counter() + for _history, counter in self.next_counts.items(): + counts.update(counter) + grand = sum(counts.values()) + if grand == 0: + uniform = 1.0 / len(self.alphabet) + return dict.fromkeys(self.alphabet, uniform) + return {symbol: counts.get(symbol, 0) / grand for symbol in self.alphabet} + + def restricted_to(self, histories: set[History]) -> SuffixCounts: + """Return a plain :class:`SuffixCounts` proxy limited to ``histories``. + + The morph/comparison helpers only read the history sets handed to them, so + stack inference can reuse them by projecting its configuration counts onto a + flat proxy without changing any results. + """ + proxy = SuffixCounts(alphabet=self.alphabet) + proxy.history_counts = Counter({h: self.history_counts.get(h, 0) for h in histories}) + proxy.next_counts = defaultdict(Counter) + for history in histories: + proxy.next_counts[history] = self.next_counts.get(history, Counter()) + return proxy + + +@dataclass +class JointSuffixCounts: + """Empirical counts of joint pasts and following input-conditioned outputs.""" + + input_alphabet: tuple[Any, ...] + output_alphabet: tuple[Any, ...] + history_counts: Counter[JointHistory] = field(default_factory=Counter) + #: ``next_counts[history][input]`` is a Counter over following output symbols. + next_counts: dict[JointHistory, dict[Any, Counter[Any]]] = field(default_factory=dict) + + @classmethod + def from_sequences( + cls, + inputs: Sequence[Any], + outputs: Sequence[Any], + *, + input_alphabet: Sequence[Any] | None = None, + output_alphabet: Sequence[Any] | None = None, + max_length: int, + ) -> JointSuffixCounts: + xs = tuple(inputs) + ys = tuple(outputs) + if len(xs) != len(ys): + raise ValueError("inputs and outputs must have equal length") + if not xs: + raise ValueError("sequences must be non-empty") + counts = cls( + input_alphabet=infer_alphabet(xs, input_alphabet), output_alphabet=infer_alphabet(ys, output_alphabet) + ) + for t, history in iter_suffixes(tuple(zip(xs, ys, strict=True)), max_length): + counts.history_counts[history] += 1 + counts.next_counts.setdefault(history, {}).setdefault(xs[t], Counter())[ys[t]] += 1 + return counts + + def output_counts(self, histories: set[JointHistory], input_symbol: Any) -> Counter[Any]: + observed: Counter[Any] = Counter() + for history in histories: + by_input = self.next_counts.get(history) + if by_input is None: + continue + counter = by_input.get(input_symbol) + if counter is not None: + observed.update(counter) + return observed + + def state_morph(self, histories: set[JointHistory], input_symbol: Any) -> dict[Any, float]: + """Return ``P(output | histories, input_symbol)``.""" + observed = self.output_counts(histories, input_symbol) + total = sum(observed.values()) + if total == 0: + return {} + return {symbol: observed.get(symbol, 0) / total for symbol in self.output_alphabet} + + +#: Aggregated output counts of a state: ``agg[input_symbol]`` is a Counter over outputs. +StateAggregate = dict[Any, Counter[Any]] + + +def _history_aggregate(counts: JointSuffixCounts, history: JointHistory) -> StateAggregate: + return {input_symbol: Counter(counter) for input_symbol, counter in counts.next_counts.get(history, {}).items()} + + +def _merge_aggregate(target: StateAggregate, source: StateAggregate) -> None: + for input_symbol, counter in source.items(): + target.setdefault(input_symbol, Counter()).update(counter) + + +def _state_aggregate(counts: JointSuffixCounts, histories: Iterable[JointHistory]) -> StateAggregate: + aggregate: StateAggregate = {} + for history in histories: + _merge_aggregate(aggregate, _history_aggregate(counts, history)) + return aggregate + + +class StackSuffixCounts(SuffixCounts): + """Empirical counts of (suffix, stack) histories and following symbols. + + Shares the morph / comparison machinery of :class:`SuffixCounts`; only the + empty-history key and the sequence-scanning constructor differ. + """ + + empty_history: ClassVar[History] = ((), ()) + + def __init__( + self, + alphabet: tuple[Any, ...], + history_counts: Counter[ConfigurationHistory] | None = None, + next_counts: dict[ConfigurationHistory, Counter[Any]] | None = None, + ) -> None: + super().__init__( + alphabet=alphabet, + history_counts=history_counts if history_counts is not None else Counter(), + next_counts=next_counts if next_counts is not None else defaultdict(Counter), + ) + + @classmethod + def from_sequence( # type: ignore[override] + cls, + sequence: Sequence[Any], + *, + alphabet: DyckAlphabet, + max_length: int | None = None, + max_stack_depth: int = 8, + ) -> StackSuffixCounts: + seq = tuple(sequence) + if not seq: + raise ValueError("sequence must be non-empty") + stacks: list[tuple[Any, ...]] = [] + stack: tuple[Any, ...] = () + for symbol in seq: + if symbol not in alphabet.symbol_alphabet: + raise ValueError(f"symbol {symbol!r} not in Dyck alphabet") + stacks.append(stack) + stack = _push(stack, symbol, alphabet=alphabet, max_stack_depth=max_stack_depth) + counts = cls(alphabet=infer_alphabet(alphabet.symbol_alphabet, None)) + for t, suffix in iter_suffixes(seq, max_length if max_length is not None else len(seq)): + history = (suffix, stacks[t]) + counts.history_counts[history] += 1 + counts.next_counts[history][seq[t]] += 1 + return counts + + +def _push(stack: tuple[Any, ...], symbol: Any, *, alphabet: DyckAlphabet, max_stack_depth: int) -> tuple[Any, ...]: + """The stack after ``symbol``: calls push (dropping the bottom past ``max_stack_depth``), returns pop.""" + if symbol in alphabet.call_alphabet: + return (*(stack[1:] if len(stack) >= max_stack_depth else stack), symbol) + if symbol in alphabet.return_alphabet: + return stack[:-1] + return stack diff --git a/sofic/inference/cssr/process.py b/sofic/inference/cssr/process.py new file mode 100644 index 0000000..16f6185 --- /dev/null +++ b/sofic/inference/cssr/process.py @@ -0,0 +1,421 @@ +"""Causal-State Splitting Reconstruction (CSSR) of ε-machines from a sample. + +CSSR follows Shalizi, Shalizi & Crutchfield (arXiv:cs/0210025). +""" + +from __future__ import annotations + +from collections import Counter, defaultdict +from collections.abc import Iterable, Mapping, Sequence +from typing import Any, Literal + +import numpy as np + +from sofic.exceptions import StochasticValidationError +from sofic.generators.epsilon_machine import EpsilonMachine +from sofic.graph import ATTR_EMISSION, ATTR_PROB, TransitionGraph +from sofic.inference.cssr.counts import History, SuffixCounts +from sofic.inference.cssr.significance import ( + MorphTest, + _bonferroni_alpha, + _observed_counts_for_morph, + morph_test_score, + morphs_differ, +) + + +def _default_max_history(n: int, alphabet_size: int) -> int: + """A third of ``log_k n``, between 1 and 10. + + Each length-``L`` word is then seen about ``n ** (2/3)`` times. Longer suffixes + multiply the number of significance tests, and with them the false splits. + """ + k = max(2, alphabet_size) + return max(1, min(10, int(np.log(n) / (3 * np.log(k))))) + + +def _suffix_homogenize( + counts: SuffixCounts, + *, + max_history: int, + alpha: float, + test: MorphTest, + min_count: int = 1, +) -> list[set[History]]: + """CSSR homogenization: grow suffixes one symbol into the past, up to length ``max_history``. + + Each child suffix ``a x`` stays in its parent's state unless its next-symbol + distribution differs significantly; then it joins the most similar state that + does not differ, or starts a new one. As in :cite:`Shalizi2002`, states keep + the suffixes of every length they collect. Suffixes seen fewer than ``min_count`` + times are not tested: the significance test is unreliable on so few counts. + """ + states: list[set[History]] = [{()}] + for length in range(max_history): + for parent_id in range(len(states)): + parent = states[parent_id] + for history in sorted((h for h in parent if len(h) == length), key=repr): + for symbol in counts.alphabet: + child = (symbol, *history) + if sum(counts.next_counts.get(child, Counter()).values()) < max(1, min_count): + continue + if not morphs_differ(counts, parent, {child}, alpha=alpha, test=test): + parent.add(child) + continue + best_id, best_score = None, float("inf") + for candidate_id, candidate in enumerate(states): + if candidate_id == parent_id: + continue + if morphs_differ(counts, candidate, {child}, alpha=alpha, test=test): + continue + score = morph_test_score(counts, candidate, {child}, test=test) + if score < best_score: + best_id, best_score = candidate_id, score + if best_id is None: + states.append({child}) + else: + states[best_id].add(child) + return states + + +def _suffix_successor(history: History, symbol: Any, max_history: int) -> History: + """The suffix that follows ``history`` on ``symbol``: extended, or truncated at ``max_history``.""" + extended = (*history, symbol) + return extended[1:] if len(extended) > max_history else extended + + +def _suffix_edges( + states: list[set[History]], + counts: SuffixCounts, + alive: set[int], + *, + max_history: int, + alpha: float, + test: MorphTest, + resolve: bool = True, +) -> dict[int, dict[Any, dict[int, set[History]]]]: + """Successor states of each alive state, by symbol, with the suffixes that lead there. + + A suffix shorter than ``max_history`` moves to the state holding its one-symbol extension. + A length-``max_history`` suffix must drop its oldest symbol, which can forget the phase + of a non-Markovian process: for the even process, the truncation of ``0111`` is + ``111``, whose parity is unknown. So the length-``max_history + 1`` suffix is tested + against the truncated suffix's state, and if its morph differs, it moves to the + alive state whose morph it matches best instead. ``resolve=False`` always truncates. + """ + history_to_state = {h: index for index in alive for h in states[index]} + edges: dict[int, dict[Any, dict[int, set[History]]]] = {} + for index in alive: + by_symbol: dict[Any, dict[int, set[History]]] = defaultdict(lambda: defaultdict(set)) + for history in states[index]: + for symbol, count in counts.next_counts.get(history, Counter()).items(): + if count == 0: + continue + extended = (*history, symbol) + if len(extended) <= max_history: + target = history_to_state.get(extended) + else: + target = history_to_state.get(extended[1:]) + if ( + resolve + and counts.next_counts.get(extended) + and ( + target is None or morphs_differ(counts, states[target], {extended}, alpha=alpha, test=test) + ) + ): + best_score = float("inf") + for candidate in sorted(alive): + if morphs_differ(counts, states[candidate], {extended}, alpha=alpha, test=test): + continue + score = morph_test_score(counts, states[candidate], {extended}, test=test) + if score < best_score: + target, best_score = candidate, score + if target is not None: + by_symbol[symbol][target].add(history) + edges[index] = by_symbol + return edges + + +def _recurrent_states(edges: Mapping[int, Mapping[Any, Iterable[int]]]) -> list[set[int]]: + """Closed communicating classes (with at least one edge) of the state graph. + + ``edges[state][label]`` iterates over the successor states on ``label`` (a + symbol here, an ``(input, output)`` pair in transCSSR). + """ + import networkx as nx + + graph = nx.DiGraph() + graph.add_nodes_from(edges) + for source, by_symbol in edges.items(): + for targets in by_symbol.values(): + graph.add_edges_from((source, target) for target in targets) + condensed = nx.condensation(graph) + classes = [] + for node in condensed: + members = set(condensed.nodes[node]["members"]) + if condensed.out_degree(node) == 0 and graph.subgraph(members).number_of_edges() > 0: + classes.append(members) + return classes + + +def _suffix_determinize( + states: list[set[History]], + counts: SuffixCounts, + alive: set[int], + *, + max_history: int, + alpha: float, + test: MorphTest, + resolve: bool = True, +) -> tuple[list[set[History]], set[int]]: + """Split alive states until each (state, symbol) pair has a single alive successor. + + Successors in pruned (transient) states are ignored, as in :cite:`Shalizi2002`. + """ + states = [set(h) for h in states] + alive = set(alive) + while True: + edges = _suffix_edges(states, counts, alive, max_history=max_history, alpha=alpha, test=test, resolve=resolve) + split = None + for index in sorted(alive): + for symbol in sorted(edges[index], key=repr): + if len(edges[index][symbol]) > 1: + split = (index, symbol) + break + if split: + break + if split is None: + return states, alive + index, symbol = split + groups = sorted(edges[index][symbol].values(), key=lambda g: (-len(g), sorted(map(repr, g)))) + for group in groups[1:]: + states[index] -= group + states.append(set(group)) + alive.add(len(states) - 1) + + +def _suffix_machine( + states: list[set[History]], + counts: SuffixCounts, + sequence: Sequence[Any], + alive: set[int], + *, + max_history: int, + alpha: float, + test: MorphTest, + resolve: bool = True, +) -> EpsilonMachine: + """Build the ε-machine on the most-visited recurrent class of the alive states.""" + edges = _suffix_edges(states, counts, alive, max_history=max_history, alpha=alpha, test=test, resolve=resolve) + history_to_state = {h: index for index in alive for h in states[index]} + + visits: Counter[int] = Counter() + seq = tuple(sequence) + for t in range(len(seq) + 1): + for length in range(min(t, max_history), -1, -1): + state = history_to_state.get(seq[t - length : t]) + if state is not None: + visits[state] += 1 + break + + classes = _recurrent_states(edges) + if not classes: + raise StochasticValidationError("no recurrent inferred states; the sample is too short for this max_history") + keep = max(classes, key=lambda members: (sum(visits[s] for s in members), -min(members))) + + labels = {state: f"s{rank}" for rank, state in enumerate(sorted(keep))} + transitions = TransitionGraph() + for state in sorted(keep): + transitions.add_state(labels[state]) + for state in sorted(keep): + longest = max(len(h) for h in states[state]) + observed = _observed_counts_for_morph(counts, {h for h in states[state] if len(h) == longest}) + weights = { + symbol: (next(iter(targets)), float(observed.get(symbol, 0))) + for symbol, targets in edges[state].items() + if observed.get(symbol, 0) > 0 + } + if not weights: + weights = { + symbol: (next(iter(targets)), float(sum(len(h) for h in targets.values()))) + for symbol, targets in edges[state].items() + } + total = sum(weight for _, weight in weights.values()) + for symbol, (target, weight) in sorted(weights.items(), key=lambda item: repr(item[0])): + transitions.add_transition( + labels[state], labels[target], **{ATTR_PROB: weight / total, ATTR_EMISSION: symbol} + ) + + kept_visits = {state: visits[state] for state in keep if visits[state] > 0} + total_visits = float(sum(kept_visits.values())) + initial = ( + {labels[state]: count / total_visits for state, count in kept_visits.items()} + if total_visits > 0 + else {labels[min(keep)]: 1.0} + ) + machine = EpsilonMachine( + graph=transitions, + initial_distribution=initial, + observation_alphabet=frozenset(counts.alphabet), + ) + machine.validate() + return machine + + +def _suffix_reconstruct( + states: list[set[History]], + counts: SuffixCounts, + sequence: Sequence[Any], + *, + max_history: int, + alpha: float, + test: MorphTest, +) -> EpsilonMachine: + """Prune transient states, determinize, and build the machine from homogeneous ``states``.""" + + def reconstruct(resolve: bool) -> EpsilonMachine: + everything = set(range(len(states))) + edges = _suffix_edges( + states, counts, everything, max_history=max_history, alpha=alpha, test=test, resolve=resolve + ) + alive = set().union(*_recurrent_states(edges)) or everything + split, alive = _suffix_determinize( + states, counts, alive, max_history=max_history, alpha=alpha, test=test, resolve=resolve + ) + return _suffix_machine( + split, counts, sequence, alive, max_history=max_history, alpha=alpha, test=test, resolve=resolve + ) + + machine = reconstruct(resolve=True) + # Resolving truncated successors needs max_history at least the synchronization length. When it + # is shorter, resolution can close off a state that never emits some observed symbol. + if {t.data[ATTR_EMISSION] for t in machine.transitions()} < set(sequence): + machine = reconstruct(resolve=False) + return machine + + +def suggest_max_history( + sequence: Sequence[Any], + *, + alpha: float = 0.01, + max_order: int | None = None, + method: Literal["exact", "chi2", "aic", "bic"] = "exact", + n_surrogates: int = 999, + seed: int = 0, +) -> int: + """A data-driven ``max_history`` for :func:`learn_epsilon_machine_cssr`: the estimated Markov order, at least 1. + + Orders ``0, 1, ...`` are tested against the next order with + :func:`dit.inference.select_markov_order`. The default ``"exact"`` method + compares the conditional block entropy against surrogates that preserve the + observed ``(order + 1)``-gram counts, which is valid at any sample size, unlike + the asymptotic chi-squared test :cite:`Pethel2014`. + + Parameters + ---------- + sequence + Observed symbols. + alpha + Significance level of each order test. + max_order + Largest order considered; by default the largest ``L`` whose + ``(L + 1)``-words are seen about 5 times each on average, at most 10. + method + ``"exact"`` or ``"chi2"`` (sequential tests), or ``"aic"`` / ``"bic"``. + n_surrogates + Surrogates per test for ``"exact"``. + seed + Seed for the surrogates, so the suggestion is reproducible. + + Notes + ----- + For a Markov source this recovers its order, which is the synchronization + length CSSR needs. A strictly sofic source (such as the even process) has + infinite Markov order, so the suggestion keeps growing with the sample; treat it + as a lower bound on the history length the data can support, not as the source's + synchronization length. + """ + import dit.inference + + select_markov_order = getattr(dit.inference, "select_markov_order", None) + if select_markov_order is None: # pragma: no cover - depends on the installed dit + raise ImportError("suggest_max_history requires a dit release with dit.inference.select_markov_order") + codes: dict[Any, str] = {} + seq = [codes.setdefault(symbol, str(len(codes))) for symbol in sequence] + if max_order is None: + k = max(2, len(set(seq))) + max_order = max(1, min(10, int(np.log(max(len(seq), 1) / 5) / np.log(k)) - 1)) + order = select_markov_order(seq, max_order, method=method, alpha=alpha, n_surrogates=n_surrogates, prng=seed) + return max(1, int(order)) + + +def learn_epsilon_machine_cssr( + sequence: Sequence[Any], + *, + alphabet: Sequence[Any] | None = None, + max_history: int | Literal["auto"] | None = None, + alpha: float = 0.01, + test: MorphTest = "g", + min_count: int = 5, + correction: Literal["bonferroni"] | None = None, +) -> EpsilonMachine: + """Reconstruct an ε-machine by Causal-State Splitting Reconstruction :cite:`Shalizi2004`. + + Suffixes are grown one symbol into the past up to length ``max_history`` and grouped by + their next-symbol distributions (homogenization), then states are split until + every transition is deterministic (determinization). The result is restricted + to its most-visited closed class, so it is always a valid recurrent machine. + + Parameters + ---------- + sequence + Observed symbols. + alphabet + Symbol alphabet; defaults to the symbols in ``sequence``. + max_history + Longest suffix considered; by default a third of ``log_k len(sequence)``, + between 1 and 10. It should be at least the synchronization length of the + source (for a Markov source, its order). Larger values run many more + significance tests, and some of them split states by chance. ``"auto"`` + uses :func:`suggest_max_history`, the Markov order estimated by exact tests. + alpha + Significance level of each morph-equality test. The worked example of + :cite:`Shalizi2002` uses 0.01; smaller values guard against spurious states + when ``max_history`` is large. + test + ``"g"`` (G-test), ``"chi2"``, ``"tv"`` (total-variation threshold), or + ``"exact"`` (Monte Carlo exact G-test when expected counts are small; see + :func:`morphs_differ`). + min_count + Suffixes seen fewer than this many times are not tested or placed in a state. + correction + ``"bonferroni"`` divides ``alpha`` by the number of suffixes eligible for + testing, bounding the chance of any spurious split. CSSR decides each test + in light of earlier ones, so step-up procedures that control the false + discovery rate (Benjamini–Hochberg) do not apply directly. + + Notes + ----- + A process that is not exactly synchronizable (no finite past determines its + state, such as :func:`~sofic.examples.alternating_biased_coins`) has no finite-``max_history`` + reconstruction. CSSR then returns more states than the ε-machine, with an + entropy rate that approaches the true one from above as ``max_history`` grows. + """ + seq = tuple(sequence) + if len(seq) < 2: + raise ValueError("sequence must contain at least two symbols") + alphabet_size = len(set(seq)) if alphabet is None else len(tuple(alphabet)) + if max_history == "auto": + max_length = suggest_max_history(seq, alpha=alpha) + else: + max_length = max_history if max_history is not None else _default_max_history(len(seq), alphabet_size) + if max_length < 0: + raise ValueError("max_history must be non-negative") + counts = SuffixCounts.from_sequence(seq, alphabet=alphabet, max_length=max_length + 1) + if correction == "bonferroni": + alpha = _bonferroni_alpha(counts, alpha, max_length=max_length, min_count=min_count) + elif correction is not None: + raise ValueError(f"unknown correction {correction!r}") + + homogeneous = _suffix_homogenize(counts, max_history=max_length, alpha=alpha, test=test, min_count=min_count) + return _suffix_reconstruct(homogeneous, counts, seq, max_history=max_length, alpha=alpha, test=test) diff --git a/sofic/inference/cssr/significance.py b/sofic/inference/cssr/significance.py new file mode 100644 index 0000000..398164b --- /dev/null +++ b/sofic/inference/cssr/significance.py @@ -0,0 +1,298 @@ +"""Significance tests that decide whether CSSR histories share a morph. + +Two-row contingency tables of next-symbol counts are compared by a G-test, +Pearson chi-squared test, or Monte Carlo exact G-test. Shared by process CSSR +(:mod:`sofic.inference.cssr.process`), stack CSSR (:mod:`sofic.inference.cssr.stack`), +and transCSSR (:mod:`sofic.inference.cssr.transducer`). +""" + +from __future__ import annotations + +import zlib +from collections import Counter +from collections.abc import Callable, Mapping, Sequence +from functools import lru_cache +from typing import Any, Literal + +import numpy as np +from scipy import stats + +from sofic.inference.cssr.counts import History, StateAggregate, SuffixCounts + +#: Contingency-table tests: G-test, Pearson chi-squared, or Monte Carlo exact G-test. +TableTest = Literal["g", "chi2", "exact"] + +#: Morph-equality tests: G-test, chi-squared, total-variation threshold, or Monte Carlo exact G-test. +MorphTest = Literal["g", "chi2", "tv", "exact"] + + +#: Monte Carlo tables drawn per ``"exact"`` test. +EXACT_DRAWS = 999 + +#: Smallest expected count at which the ``"exact"`` test trusts the chi-squared limit. +EXACT_MIN_EXPECTED = 5.0 + + +def contingency_table(left: Mapping[Any, int], right: Mapping[Any, int], alphabet: Sequence[Any]) -> np.ndarray | None: + """Two-row table of ``left`` and ``right`` counts over the symbols either one observed. + + Returns ``None`` when there is nothing to test: no observed symbol, identical + rows, or a single symbol with an empty row or equal proportions. + """ + active = [symbol for symbol in alphabet if left.get(symbol, 0) + right.get(symbol, 0) > 0] + if not active: + return None + table = np.array( + [[left.get(symbol, 0) for symbol in active], [right.get(symbol, 0) for symbol in active]], + dtype=float, + ) + if np.allclose(table[0], table[1]): + return None + if table.shape[1] < 2: + left_total = table[0].sum() + right_total = table[1].sum() + if left_total == 0.0 or right_total == 0.0: + return None + if np.isclose(table[0, 0] / left_total, table[1, 0] / right_total): + return None + return table + + +def g_statistic(table: np.ndarray) -> float | None: + """G-test statistic of a contingency table, as ``scipy.stats.chi2_contingency`` computes it. + + Applies Yates' continuity correction when the table has one degree of freedom + (``chi2_contingency`` does so for every ``lambda_``, including + ``"log-likelihood"``), and returns ``None`` if an expected count is zero. + Inlined because the test runs once per pair of histories, and the general + scipy routine dominated inference time. + """ + expected = table.sum(axis=1, keepdims=True) * table.sum(axis=0, keepdims=True) / table.sum() + if np.any(expected == 0): + return None + observed = table + if (table.shape[0] - 1) * (table.shape[1] - 1) == 1: + diff = expected - observed + observed = observed + np.sign(diff) * np.minimum(0.5, np.abs(diff)) + with np.errstate(divide="ignore", invalid="ignore"): + terms = np.where(observed > 0, observed * np.log(observed / expected), 0.0) + return 2.0 * float(terms.sum()) + + +@lru_cache(maxsize=256) +def chi2_critical(alpha: float, dof: int) -> float: + """Upper ``alpha`` quantile of the chi-squared distribution with ``dof`` degrees of freedom.""" + return float(stats.chi2.ppf(1.0 - alpha, dof)) + + +def g_significant(table: np.ndarray, alpha: float) -> bool: + """Asymptotic G-test (see :func:`g_statistic`) at level ``alpha``.""" + statistic = g_statistic(table) + if statistic is None or not np.isfinite(statistic): + return False + return statistic > chi2_critical(alpha, max(1, table.shape[1] - 1)) + + +def chi2_significant(table: np.ndarray, alpha: float) -> bool: + """Pearson chi-squared test (with Yates' correction at one degree of freedom) at level ``alpha``.""" + try: + _statistic, p_value, _dof, expected = stats.chi2_contingency(table) + except ValueError: + return False + if np.any(expected == 0): + return False + return float(p_value) < alpha + + +def exact_g_pvalue(table: np.ndarray) -> float: + """Monte Carlo p-value of the G statistic among tables with the same margins. + + Tables are drawn uniformly given both margins (``scipy.stats.random_table``), + the exact null of equal morphs. The generator is seeded from the table itself, + so reconstruction stays deterministic. + """ + counts = table.astype(np.int64) + rows, cols = counts.sum(axis=1), counts.sum(axis=0) + expected = np.outer(rows, cols) / counts.sum() + + def g(observed: np.ndarray) -> np.ndarray: + with np.errstate(divide="ignore", invalid="ignore"): + terms = np.where(observed > 0, observed * np.log(observed / expected), 0.0) + return 2.0 * terms.sum(axis=(-2, -1)) + + rng = np.random.default_rng(zlib.crc32(counts.tobytes())) + draws = stats.random_table(rows, cols, seed=rng).rvs(size=EXACT_DRAWS) + observed = g(counts.astype(float)) + extreme = np.sum(g(draws.astype(float)) >= observed - 1e-9 * max(1.0, observed)) + return float((1 + extreme) / (1 + EXACT_DRAWS)) + + +def exact_significant(table: np.ndarray, alpha: float) -> bool: + """The ``"exact"`` decision for a contingency table of next-symbol counts. + + Uses the Monte Carlo exact G-test when an expected count is below + :data:`EXACT_MIN_EXPECTED` and the asymptotic G-test otherwise. + """ + if np.any(table.sum(axis=1) == 0): + return False + expected = np.outer(table.sum(axis=1), table.sum(axis=0)) / table.sum() + if expected.min() < EXACT_MIN_EXPECTED: + return exact_g_pvalue(table) < alpha + return g_significant(table, alpha) + + +def table_significant(table: np.ndarray, alpha: float, test: TableTest) -> bool: + """Whether the rows of ``table`` differ significantly at level ``alpha`` under ``test``.""" + if test == "exact": + return exact_significant(table, alpha) + if test == "g": + return g_significant(table, alpha) + return chi2_significant(table, alpha) + + +def table_score(table: np.ndarray, test: TableTest) -> float: + """Test statistic of ``table`` (lower is more similar): G for ``"g"``/``"exact"``, else Pearson.""" + if test == "chi2": + try: + statistic, _p, _dof, _expected = stats.chi2_contingency(table) + except ValueError: + return 0.0 + return float(statistic) + statistic = g_statistic(table) + return statistic if statistic is not None and np.isfinite(statistic) else 0.0 + + +def _observed_counts_for_morph( + counts: SuffixCounts, + histories: set[History], +) -> Counter[Any]: + observed = Counter() + for history in histories: + observed.update(counts.next_counts.get(history, Counter())) + return observed + + +def _contingency_rows( + counts: SuffixCounts, + left_histories: set[History], + right_histories: set[History], +) -> np.ndarray | None: + return contingency_table( + _observed_counts_for_morph(counts, left_histories), + _observed_counts_for_morph(counts, right_histories), + counts.alphabet, + ) + + +def _bonferroni_alpha( + counts: SuffixCounts, + alpha: float, + *, + max_length: int, + min_count: int, + suffix_length: Callable[[History], int] = len, +) -> float: + """``alpha`` divided by the number of suffixes eligible for a split test.""" + eligible = sum( + 1 + for history, following in counts.next_counts.items() + if 0 < suffix_length(history) <= max_length and sum(following.values()) >= max(1, min_count) + ) + return alpha / max(1, eligible) + + +def morphs_differ( + counts: SuffixCounts, + left_histories: set[History], + right_histories: set[History], + *, + alpha: float = 0.05, + test: MorphTest = "g", + delta: float = 0.0, +) -> bool: + """Return whether two history sets have significantly different morphs. + + ``"g"`` is the G-test (log-likelihood ratio) with Yates' continuity correction + when the table has one degree of freedom, i.e. two observed symbols; this is + the statistic ``scipy.stats.chi2_contingency(table, lambda_="log-likelihood")`` + reports. transCSSR uses the same test. ``"g"`` and ``"chi2"`` use the + chi-squared limit, which is unreliable when expected counts are small. + ``"exact"`` instead compares the G statistic with tables drawn uniformly given + the observed margins whenever an expected count is below 5, and uses the + G-test otherwise. + """ + if test == "tv": + left = counts.state_morph(left_histories) + right = counts.state_morph(right_histories) + distance = 0.5 * sum(abs(left[s] - right[s]) for s in counts.alphabet) + return distance > delta + + table = _contingency_rows(counts, left_histories, right_histories) + if table is None: + return False + return table_significant(table, alpha, test) + + +def morph_test_score( + counts: SuffixCounts, + left_histories: set[History], + right_histories: set[History], + *, + test: MorphTest = "g", +) -> float: + """Score for matching morphs (lower is more similar).""" + if test == "tv": + left = counts.state_morph(left_histories) + right = counts.state_morph(right_histories) + return 0.5 * sum(abs(left[s] - right[s]) for s in counts.alphabet) + table = _contingency_rows(counts, left_histories, right_histories) + if table is None: + return 0.0 + return table_score(table, test) + + +def aggregates_differ( + left: StateAggregate, + right: StateAggregate, + *, + input_alphabet: tuple[Any, ...], + output_alphabet: tuple[Any, ...], + alpha: float, + test: TableTest = "g", +) -> bool: + """Return whether two aggregated morphs differ on ``P(output | ., input)`` for some input. + + Each input symbol's output counts are compared with the same test as process + CSSR (:func:`~sofic.inference.cssr.morphs_differ`); in particular + ``"g"`` is the G-test with Yates' continuity correction at one degree of freedom. + """ + for input_symbol in input_alphabet: + table = contingency_table( + left.get(input_symbol, Counter()), + right.get(input_symbol, Counter()), + output_alphabet, + ) + if table is None: + continue + if table_significant(table, alpha, test): + return True + return False + + +def _aggregate_score( + left: StateAggregate, + right: StateAggregate, + *, + input_alphabet: tuple[Any, ...], + output_alphabet: tuple[Any, ...], +) -> float: + total = 0.0 + for input_symbol in input_alphabet: + table = contingency_table( + left.get(input_symbol, Counter()), + right.get(input_symbol, Counter()), + output_alphabet, + ) + if table is not None: + total += table_score(table, "g") + return total diff --git a/sofic/generators/stack_inference.py b/sofic/inference/cssr/stack.py similarity index 71% rename from sofic/generators/stack_inference.py rename to sofic/inference/cssr/stack.py index 6028214..1d8d605 100644 --- a/sofic/generators/stack_inference.py +++ b/sofic/inference/cssr/stack.py @@ -3,95 +3,26 @@ from __future__ import annotations from collections import Counter, defaultdict -from collections.abc import Callable, Hashable, Sequence -from typing import Any, ClassVar, Literal +from collections.abc import Callable, Hashable, Mapping, Sequence +from typing import Any, Literal -from sofic.automata.papni import DyckAlphabet, is_well_matched, learn_sofic_dyck_shift_papni +from sofic.automata.learning.papni import DyckAlphabet, is_well_matched, learn_sofic_dyck_shift_papni from sofic.exceptions import StochasticValidationError -from sofic.generators.epsilon_inference import ( - History, - MorphTest, - SuffixCounts, - _bonferroni_alpha, - _cluster_histories_by_morph, - _cssr_default_lmax, - _cssr_determinize, - _drop_transient_states, - _merge_similar_states, - morph_test_score, - morphs_differ, - suggest_lmax, -) from sofic.generators.stack_hmm import HiddenMarkovStackModel from sofic.graph import ATTR_SYMBOL +from sofic.inference.cssr.counts import ConfigurationHistory, StackSuffixCounts, _push +from sofic.inference.cssr.process import _default_max_history, _recurrent_states, suggest_max_history +from sofic.inference.cssr.significance import MorphTest, _bonferroni_alpha, morph_test_score, morphs_differ +from sofic.inference.cssr.subtree import _cluster_histories_by_morph from sofic.shifts.sofic_dyck import SoficDyckShift, TransitionRef, transition_ref __all__ = [ - "ConfigurationHistory", - "StackSuffixCounts", - "fit_stack_hmm_mle", + "learn_stack_hmm_mle", "learn_stack_hmm_papni", - "stack_cssr", - "stack_subtree_merge", + "learn_stack_hmm_cssr", + "learn_stack_hmm_subtree", ] -ConfigurationHistory = tuple[tuple[Any, ...], tuple[Any, ...]] - - -class StackSuffixCounts(SuffixCounts): - """Empirical counts of (suffix, stack) histories and following symbols. - - Shares the morph / comparison machinery of :class:`SuffixCounts`; only the - empty-history key and the sequence-scanning constructor differ. - """ - - empty_history: ClassVar[History] = ((), ()) - - def __init__( - self, - alphabet: tuple[Any, ...], - history_counts: Counter[ConfigurationHistory] | None = None, - next_counts: dict[ConfigurationHistory, Counter[Any]] | None = None, - ) -> None: - super().__init__( - alphabet=alphabet, - history_counts=history_counts if history_counts is not None else Counter(), - next_counts=next_counts if next_counts is not None else defaultdict(Counter), - ) - - @classmethod - def from_sequence( # type: ignore[override] - cls, - sequence: Sequence[Any], - *, - alphabet: DyckAlphabet, - max_length: int | None = None, - max_stack_depth: int = 8, - ) -> StackSuffixCounts: - seq = tuple(sequence) - if not seq: - raise ValueError("sequence must be non-empty") - visible_alphabet = tuple(sorted(alphabet.symbol_alphabet, key=repr)) - max_len = max_length if max_length is not None else len(seq) - counts = cls(alphabet=visible_alphabet) - stack: list[Any] = [] - for t, symbol in enumerate(seq): - if symbol not in alphabet.symbol_alphabet: - raise ValueError(f"symbol {symbol!r} not in Dyck alphabet") - for length in range(0, min(t, max_len) + 1): - suffix = seq[t - length : t] - history = (suffix, tuple(stack)) - counts.history_counts[history] += 1 - counts.next_counts[history][symbol] += 1 - if symbol in alphabet.call_alphabet: - if len(stack) >= max_stack_depth: - stack = stack[1:] - stack.append(symbol) - elif symbol in alphabet.return_alphabet: - if stack: - stack.pop() - return counts - def _successor_history( history: ConfigurationHistory, @@ -109,15 +40,7 @@ def _successor_history( new_suffix = extended else: new_suffix = extended[-length:] - - stack_list = list(stack) - if symbol in alphabet.call_alphabet: - if len(stack_list) >= max_stack_depth: - stack_list = stack_list[1:] - stack_list.append(symbol) - elif symbol in alphabet.return_alphabet and stack_list: - stack_list.pop() - return new_suffix, tuple(stack_list) + return new_suffix, _push(stack, symbol, alphabet=alphabet, max_stack_depth=max_stack_depth) def _stack_successor_fn( @@ -167,7 +90,7 @@ def _stack_homogenize( counts: StackSuffixCounts, *, alphabet: DyckAlphabet, - Lmax: int, + max_history: int, alpha: float, test: MorphTest, max_stack_depth: int, @@ -178,7 +101,7 @@ def _stack_homogenize( Every observed stack contributes a root ``((), stack)``; suffixes then grow one symbol into the past with their stack fixed, exactly as in flat CSSR. Growing forward from the empty configuration instead only reaches stacks of depth at - most ``Lmax``, so deeper configurations had no state and their transitions were + most ``max_history``, so deeper configurations had no state and their transitions were dropped. Morphs are compared with return symbols collapsed (see :func:`_control_counts`). """ control = _control_counts(counts, alphabet) @@ -209,7 +132,7 @@ def observed(history: ConfigurationHistory) -> bool: for stack in sorted(roots, key=lambda stack: (len(stack), repr(stack))): if observed(((), stack)): place(((), stack), 0) - for length in range(Lmax): + for length in range(max_history): for state_id in sorted(states): for suffix, stack in sorted((h for h in states[state_id] if len(h[0]) == length), key=repr): for symbol in counts.alphabet: @@ -228,14 +151,48 @@ def _stack_determinize( alphabet: DyckAlphabet, max_stack_depth: int, ) -> dict[int, set[ConfigurationHistory]]: - """Split homogeneous states until stack-lifted transitions are unifilar.""" - return _cssr_determinize( - states, - history_to_state, - counts, - length=length, - successor_fn=_stack_successor_fn(alphabet=alphabet, length=length, max_stack_depth=max_stack_depth), - ) + """Split homogeneous states until stack-lifted transitions are unifilar. + + Updates ``history_to_state`` in place for the histories that move. + """ + successor = _stack_successor_fn(alphabet=alphabet, length=length, max_stack_depth=max_stack_depth) + current = {state_id: set(histories) for state_id, histories in states.items()} + changed = True + next_state_id = max(current) + 1 if current else 0 + + while changed: + changed = False + for state_id in sorted(current): + histories = current[state_id] + if len(histories) <= 1: + continue + for symbol in counts.alphabet: + buckets: dict[int, set[ConfigurationHistory]] = defaultdict(set) + for history in histories: + if counts.next_counts.get(history, Counter()).get(symbol, 0) == 0: + continue + target = history_to_state.get(successor(history, symbol)) + if target is None: + continue + buckets[target].add(history) + if len(buckets) <= 1: + continue + # Keep the largest bucket -- and every history that never emits + # ``symbol`` -- in the original state; split the other buckets off. + ordered = sorted(buckets.items(), key=lambda item: (-len(item[1]), min(item[1]))) + moved = set().union(*(split for _target, split in ordered[1:])) + current[state_id] = histories - moved + for _target, split_histories in ordered[1:]: + new_id = next_state_id + next_state_id += 1 + current[new_id] = split_histories + for history in split_histories: + history_to_state[history] = new_id + changed = True + break + if changed: + break + return current def _stack_merge( @@ -247,8 +204,33 @@ def _stack_merge( test: MorphTest, alphabet: DyckAlphabet, ) -> dict[int, set[ConfigurationHistory]]: - proxy = _control_counts(counts, alphabet).restricted_to(set(history_to_state)) - return _merge_similar_states(states, history_to_state, proxy, alpha=alpha, test=test) + """Merge states whose pooled control morphs (see :func:`_control_counts`) are indistinguishable. + + Merging on the morph alone can fuse states with incompatible ``symbol -> + successor`` maps, so the result need not be unifilar. + """ + control = _control_counts(counts, alphabet) + current = {state_id: set(histories) for state_id, histories in states.items()} + changed = True + while changed: + changed = False + state_ids = sorted(current) + for index, left_id in enumerate(state_ids): + if left_id not in current: + continue + for right_id in state_ids[index + 1 :]: + if right_id not in current: + continue + if morphs_differ(control, current[left_id], current[right_id], alpha=alpha, test=test): + continue + current[left_id].update(current.pop(right_id)) + for history in current[left_id]: + history_to_state[history] = left_id + changed = True + break + if changed: + break + return current def _stack_drop_transient( @@ -260,14 +242,32 @@ def _stack_drop_transient( alphabet: DyckAlphabet, max_stack_depth: int, ) -> dict[int, set[ConfigurationHistory]]: - proxy = counts.restricted_to(set(history_to_state)) - return _drop_transient_states( - states, - history_to_state, - proxy, - length=length, - successor_fn=_stack_successor_fn(alphabet=alphabet, length=length, max_stack_depth=max_stack_depth), - ) + """Keep only states in closed communicating classes; all states if there are none. + + States left with no edge into the kept set (their successor configurations + were too rare to be placed) are pruned repeatedly, since a generator state + needs outgoing mass. + """ + successor = _stack_successor_fn(alphabet=alphabet, length=length, max_stack_depth=max_stack_depth) + + def state_edges(kept: Mapping[int, set[ConfigurationHistory]]) -> dict[int, dict[Any, set[int]]]: + edges: dict[int, dict[Any, set[int]]] = {state_id: defaultdict(set) for state_id in kept} + for state_id, histories in kept.items(): + for history in histories: + for symbol, count in counts.next_counts.get(history, Counter()).items(): + target = history_to_state.get(successor(history, symbol)) if count else None + if target in kept: + edges[state_id][symbol].add(target) + return edges + + recurrent = set().union(*_recurrent_states(state_edges(states))) + kept = {state_id: histories for state_id, histories in states.items() if not recurrent or state_id in recurrent} + while True: + edges = state_edges(kept) + dead = {state_id for state_id, by_symbol in edges.items() if not any(by_symbol.values())} + if not dead or len(dead) == len(kept): + return kept + kept = {state_id: histories for state_id, histories in kept.items() if state_id not in dead} def _counts_to_stack_hmm( @@ -282,21 +282,15 @@ def _counts_to_stack_hmm( ) -> HiddenMarkovStackModel: visits: Counter[int] = Counter() seq = tuple(sequence) - stack: list[Any] = [] + stack: tuple[Any, ...] = () for t in range(len(seq)): # Each step occupies one state: the one keyed by its longest available suffix. for hist_len in range(min(t, length), -1, -1): - state = history_to_state.get((seq[t - hist_len : t], tuple(stack))) + state = history_to_state.get((seq[t - hist_len : t], stack)) if state is not None: visits[state] += 1 break - symbol = seq[t] - if symbol in alphabet.call_alphabet: - if len(stack) >= max_stack_depth: - stack = stack[1:] - stack.append(symbol) - elif symbol in alphabet.return_alphabet and stack: - stack.pop() + stack = _push(stack, seq[t], alphabet=alphabet, max_stack_depth=max_stack_depth) if not visits: raise StochasticValidationError("no empirical configuration visits") @@ -377,11 +371,11 @@ def legal(symbol: Any, stack: tuple[Any, ...]) -> bool: return model -def stack_cssr( +def learn_stack_hmm_cssr( sequence: Sequence[Any], *, alphabet: DyckAlphabet, - Lmax: int | Literal["auto"] | None = None, + max_history: int | Literal["auto"] | None = None, max_stack_depth: int = 8, alpha: float = 0.05, test: MorphTest = "g", @@ -390,20 +384,22 @@ def stack_cssr( ) -> HiddenMarkovStackModel: """Reconstruct a stack HMM via configuration-lifted CSSR. - ``Lmax="auto"`` uses :func:`~sofic.generators.epsilon_inference.suggest_lmax` + ``max_history="auto"`` uses :func:`~sofic.inference.cssr.suggest_max_history` on the observed symbols. Stack processes generally have infinite Markov order, so treat it as a lower bound on the suffix length the data support. ``test="exact"`` and ``correction="bonferroni"`` are as in - :func:`~sofic.generators.epsilon_inference.cssr`; the correction counts + :func:`~sofic.inference.cssr.learn_epsilon_machine_cssr`; the correction counts eligible (suffix, stack) configurations. """ seq = tuple(sequence) if len(seq) < 2: raise ValueError("sequence must contain at least two symbols") - if Lmax == "auto": - max_length = suggest_lmax(seq, alpha=alpha) + if max_history == "auto": + max_length = suggest_max_history(seq, alpha=alpha) else: - max_length = Lmax if Lmax is not None else _cssr_default_lmax(len(seq), len(alphabet.symbol_alphabet)) + max_length = ( + max_history if max_history is not None else _default_max_history(len(seq), len(alphabet.symbol_alphabet)) + ) counts = StackSuffixCounts.from_sequence( seq, alphabet=alphabet, @@ -423,7 +419,7 @@ def stack_cssr( states, history_to_state = _stack_homogenize( counts, alphabet=alphabet, - Lmax=max_length, + max_history=max_length, alpha=alpha, test=test, max_stack_depth=max_stack_depth, @@ -453,27 +449,27 @@ def stack_cssr( ) -def stack_subtree_merge( +def learn_stack_hmm_subtree( sequence: Sequence[Any], *, alphabet: DyckAlphabet, - L: int, + max_history: int, max_stack_depth: int = 8, delta: float = 0.0, ) -> HiddenMarkovStackModel: - """Reconstruct a stack HMM by merging depth-``L`` configuration subtrees.""" - if L < 0: - raise ValueError("L must be non-negative") + """Reconstruct a stack HMM by merging depth-``max_history`` configuration subtrees.""" + if max_history < 0: + raise ValueError("max_history must be non-negative") seq = tuple(sequence) if len(seq) < 2: raise ValueError("sequence must contain at least two symbols") counts = StackSuffixCounts.from_sequence( seq, alphabet=alphabet, - max_length=L + 1, + max_length=max_history + 1, max_stack_depth=max_stack_depth, ) - histories = {history for history in counts.history_counts if len(history[0]) <= L} + histories = {history for history in counts.history_counts if len(history[0]) <= max_history} histories.add(((), ())) proxy = counts.restricted_to(histories) states = _cluster_histories_by_morph(proxy, histories, delta=delta) @@ -482,7 +478,7 @@ def stack_subtree_merge( states, history_to_state, counts, - length=L, + length=max_history, alphabet=alphabet, max_stack_depth=max_stack_depth, ) @@ -494,7 +490,7 @@ def stack_subtree_merge( history: state_id for state_id, histories_in_state in states.items() for history in histories_in_state } states = _stack_drop_transient( - states, history_to_state, counts, length=L, alphabet=alphabet, max_stack_depth=max_stack_depth + states, history_to_state, counts, length=max_history, alphabet=alphabet, max_stack_depth=max_stack_depth ) history_to_state = { history: state_id for state_id, histories_in_state in states.items() for history in histories_in_state @@ -505,12 +501,12 @@ def stack_subtree_merge( history_to_state, seq, alphabet=alphabet, - length=L, + length=max_history, max_stack_depth=max_stack_depth, ) -def fit_stack_hmm_mle( +def learn_stack_hmm_mle( shift: SoficDyckShift, sequence: Sequence[Any], *, @@ -574,4 +570,4 @@ def learn_stack_hmm_papni( fit_source = max((tuple(word) for word in positive if is_well_matched(word, alphabet)), key=len, default=()) if not fit_source: raise ValueError("no sequence available for parameter fitting") - return fit_stack_hmm_mle(shift, fit_source) + return learn_stack_hmm_mle(shift, fit_source) diff --git a/sofic/inference/cssr/subtree.py b/sofic/inference/cssr/subtree.py new file mode 100644 index 0000000..9278fcd --- /dev/null +++ b/sofic/inference/cssr/subtree.py @@ -0,0 +1,125 @@ +"""ε-machine reconstruction by merging depth-``L`` subtrees. + +Follows Crutchfield & Young (PRL 1989; PRE 1994). +""" + +from __future__ import annotations + +from collections import defaultdict +from collections.abc import Sequence +from typing import Any, Literal + +from sofic.generators.epsilon_machine import EpsilonMachine +from sofic.inference.cssr.counts import History, SuffixCounts +from sofic.inference.cssr.process import _suffix_reconstruct, suggest_max_history +from sofic.inference.cssr.significance import MorphTest, morphs_differ + +#: Significance level used by subtree merging when ``delta = 0`` and to resolve truncated successors. +_SUBTREE_ALPHA = 0.01 + + +def _morph_distance( + counts: SuffixCounts, + left: History, + right: History, + *, + delta: float, +) -> float: + left_morph = counts.morph(left) + right_morph = counts.morph(right) + return 0.5 * sum(abs(left_morph[s] - right_morph[s]) for s in counts.alphabet) + + +def _morphs_equivalent( + counts: SuffixCounts, + left: History, + right: History, + *, + delta: float, + alpha: float = _SUBTREE_ALPHA, + test: MorphTest = "g", +) -> bool: + if delta > 0.0: + return _morph_distance(counts, left, right, delta=delta) <= delta + return not morphs_differ(counts, {left}, {right}, alpha=alpha, test=test) + + +def _cluster_histories_by_morph( + counts: SuffixCounts, + histories: set[History], + *, + delta: float, + alpha: float = _SUBTREE_ALPHA, + test: MorphTest = "g", +) -> dict[int, set[History]]: + parent: dict[History, History] = {history: history for history in histories} + + def find(history: History) -> History: + root = history + while parent[root] != root: + parent[root] = parent[parent[root]] + root = parent[root] + return root + + def union(left: History, right: History) -> None: + left_root = find(left) + right_root = find(right) + if left_root != right_root: + parent[right_root] = left_root + + history_list = sorted(histories) + for index, left in enumerate(history_list): + for right in history_list[index + 1 :]: + if _morphs_equivalent(counts, left, right, delta=delta, alpha=alpha, test=test): + union(left, right) + + clusters: dict[History, set[History]] = defaultdict(set) + for history in histories: + clusters[find(history)].add(history) + + states: dict[int, set[History]] = {} + for state_id, (_root, members) in enumerate(clusters.items()): + states[state_id] = set(members) + return states + + +def learn_epsilon_machine_subtree( + sequence: Sequence[Any], + *, + max_history: int | Literal["auto"], + delta: float = 0.0, + alphabet: Sequence[Any] | None = None, + alpha: float = _SUBTREE_ALPHA, + test: MorphTest = "g", + correction: Literal["bonferroni"] | None = None, +) -> EpsilonMachine: + """Reconstruct an ε-machine by merging depth-``max_history`` subtrees (Crutchfield--Young). + + Histories up to length ``max_history`` are clustered by next-symbol distribution: within + total-variation distance ``delta``, or, when ``delta = 0``, unless a morph test + (``test``, at level ``alpha``) tells them apart. The clusters are then + determinized as in :func:`learn_epsilon_machine_cssr`. + + ``max_history="auto"`` uses :func:`suggest_max_history`. ``correction="bonferroni"`` divides + ``alpha`` by the number of history pairs compared, so that no pair is split + apart by chance; since a rejected test *separates* histories, this makes the + reconstruction more conservative (fewer states). + """ + if max_history == "auto": + max_history = suggest_max_history(sequence, alpha=alpha) + if max_history < 0: + raise ValueError("max_history must be non-negative") + seq = tuple(sequence) + if len(seq) < 2: + raise ValueError("sequence must contain at least two symbols") + counts = SuffixCounts.from_sequence(seq, alphabet=alphabet, max_length=max_history + 1) + + histories = {history for history in counts.history_counts if len(history) <= max_history} + histories.add(()) + if correction == "bonferroni": + alpha /= max(1, len(histories) * (len(histories) - 1) // 2) + elif correction is not None: + raise ValueError(f"unknown correction {correction!r}") + states = list(_cluster_histories_by_morph(counts, histories, delta=delta, alpha=alpha, test=test).values()) + + return _suffix_reconstruct(states, counts, seq, max_history=max_history, alpha=alpha, test=test) diff --git a/sofic/generators/epsilon_transducer_inference.py b/sofic/inference/cssr/transducer.py similarity index 57% rename from sofic/generators/epsilon_transducer_inference.py rename to sofic/inference/cssr/transducer.py index 2466b1b..e7588c5 100644 --- a/sofic/generators/epsilon_transducer_inference.py +++ b/sofic/inference/cssr/transducer.py @@ -13,187 +13,22 @@ from __future__ import annotations from collections import Counter, defaultdict -from collections.abc import Iterable, Sequence -from dataclasses import dataclass, field +from collections.abc import Sequence from typing import Any, Literal -import numpy as np -from scipy import stats - from sofic.exceptions import StochasticValidationError from sofic.generators.epsilon_transducer import EpsilonTransducer from sofic.graph import ATTR_OUTPUT, ATTR_PROB, ATTR_SYMBOL, TransitionGraph - -JointHistory = tuple[tuple[Any, Any], ...] - - -@dataclass -class JointSuffixCounts: - """Empirical counts of joint pasts and following input-conditioned outputs.""" - - input_alphabet: tuple[Any, ...] - output_alphabet: tuple[Any, ...] - history_counts: Counter[JointHistory] = field(default_factory=Counter) - #: ``next_counts[history][input]`` is a Counter over following output symbols. - next_counts: dict[JointHistory, dict[Any, Counter[Any]]] = field(default_factory=dict) - - @classmethod - def from_sequences( - cls, - inputs: Sequence[Any], - outputs: Sequence[Any], - *, - input_alphabet: Sequence[Any] | None = None, - output_alphabet: Sequence[Any] | None = None, - max_length: int, - ) -> JointSuffixCounts: - xs = tuple(inputs) - ys = tuple(outputs) - if len(xs) != len(ys): - raise ValueError("inputs and outputs must have equal length") - if not xs: - raise ValueError("sequences must be non-empty") - in_alpha = tuple(sorted(set(xs), key=repr)) if input_alphabet is None else tuple(input_alphabet) - out_alpha = tuple(sorted(set(ys), key=repr)) if output_alphabet is None else tuple(output_alphabet) - counts = cls(input_alphabet=in_alpha, output_alphabet=out_alpha) - pairs = tuple(zip(xs, ys, strict=True)) - n = len(pairs) - for t in range(n): - for length in range(0, min(t, max_length) + 1): - history = pairs[t - length : t] - counts.history_counts[history] += 1 - by_input = counts.next_counts.setdefault(history, {}) - by_input.setdefault(xs[t], Counter())[ys[t]] += 1 - return counts - - def output_counts(self, histories: set[JointHistory], input_symbol: Any) -> Counter[Any]: - observed: Counter[Any] = Counter() - for history in histories: - by_input = self.next_counts.get(history) - if by_input is None: - continue - counter = by_input.get(input_symbol) - if counter is not None: - observed.update(counter) - return observed - - def state_morph(self, histories: set[JointHistory], input_symbol: Any) -> dict[Any, float]: - """Return ``P(output | histories, input_symbol)``.""" - observed = self.output_counts(histories, input_symbol) - total = sum(observed.values()) - if total == 0: - return {} - return {symbol: observed.get(symbol, 0) / total for symbol in self.output_alphabet} - - -def _output_contingency(left: Counter[Any], right: Counter[Any], alphabet: tuple[Any, ...]) -> np.ndarray | None: - active = [symbol for symbol in alphabet if left.get(symbol, 0) + right.get(symbol, 0) > 0] - if not active: - return None - table = np.array( - [[left.get(symbol, 0) for symbol in active], [right.get(symbol, 0) for symbol in active]], - dtype=float, - ) - if np.allclose(table[0], table[1]): - return None - if table.shape[1] < 2: - left_total = table[0].sum() - right_total = table[1].sum() - if left_total == 0.0 or right_total == 0.0: - return None - if np.isclose(table[0, 0] / left_total, table[1, 0] / right_total): - return None - return table - - -#: Aggregated output counts of a state: ``agg[input_symbol]`` is a Counter over outputs. -StateAggregate = dict[Any, Counter[Any]] - - -def _history_aggregate(counts: JointSuffixCounts, history: JointHistory) -> StateAggregate: - return {input_symbol: Counter(counter) for input_symbol, counter in counts.next_counts.get(history, {}).items()} - - -def _merge_aggregate(target: StateAggregate, source: StateAggregate) -> None: - for input_symbol, counter in source.items(): - target.setdefault(input_symbol, Counter()).update(counter) - - -def aggregates_differ( - left: StateAggregate, - right: StateAggregate, - *, - input_alphabet: tuple[Any, ...], - output_alphabet: tuple[Any, ...], - alpha: float, - test: Literal["g", "chi2", "exact"] = "g", -) -> bool: - """Return whether two aggregated morphs differ on ``P(output | ., input)`` for some input.""" - for input_symbol in input_alphabet: - table = _output_contingency( - left.get(input_symbol, Counter()), - right.get(input_symbol, Counter()), - output_alphabet, - ) - if table is None: - continue - if _table_significant(table, alpha=alpha, test=test): - return True - return False - - -def _aggregate_score( - left: StateAggregate, - right: StateAggregate, - *, - input_alphabet: tuple[Any, ...], - output_alphabet: tuple[Any, ...], -) -> float: - total = 0.0 - for input_symbol in input_alphabet: - table = _output_contingency( - left.get(input_symbol, Counter()), - right.get(input_symbol, Counter()), - output_alphabet, - ) - if table is None: - continue - try: - with np.errstate(invalid="ignore", divide="ignore"): - statistic, _p, _dof, _expected = stats.chi2_contingency(table, lambda_="log-likelihood") - if np.isfinite(statistic): - total += float(statistic) - except ValueError: - continue - return total - - -def _table_significant(table: np.ndarray, *, alpha: float, test: Literal["g", "chi2", "exact"]) -> bool: - if test == "exact": - from sofic.generators.epsilon_inference import _exact_significant - - return _exact_significant(table, alpha) - try: - if test == "g": - with np.errstate(invalid="ignore", divide="ignore"): - statistic, _p, _dof, expected = stats.chi2_contingency(table, lambda_="log-likelihood") - if not np.isfinite(statistic) or np.any(expected == 0): - return False - dof = max(1, table.shape[1] - 1) - return float(statistic) > float(stats.chi2.ppf(1.0 - alpha, dof)) - statistic, p_value, _dof, expected = stats.chi2_contingency(table) - except ValueError: - return False - if np.any(expected == 0): - return False - return float(p_value) < alpha - - -def _state_aggregate(counts: JointSuffixCounts, histories: Iterable[JointHistory]) -> StateAggregate: - aggregate: StateAggregate = {} - for history in histories: - _merge_aggregate(aggregate, _history_aggregate(counts, history)) - return aggregate +from sofic.inference.cssr.counts import ( + JointHistory, + JointSuffixCounts, + StateAggregate, + _history_aggregate, + _merge_aggregate, + _state_aggregate, +) +from sofic.inference.cssr.process import _recurrent_states, suggest_max_history +from sofic.inference.cssr.significance import TableTest, _aggregate_score, aggregates_differ def _observed(counts: JointSuffixCounts, history: JointHistory) -> int: @@ -203,9 +38,9 @@ def _observed(counts: JointSuffixCounts, history: JointHistory) -> int: def _homogenize( counts: JointSuffixCounts, *, - Lmax: int, + max_history: int, alpha: float, - test: Literal["g", "chi2", "exact"], + test: TableTest, min_count: int, ) -> list[set[JointHistory]]: """transCSSR homogenization: grow joint suffixes one ``(input, output)`` pair into the past. @@ -224,7 +59,7 @@ def differ(left: StateAggregate, right: StateAggregate) -> bool: left, right, input_alphabet=in_alpha, output_alphabet=out_alpha, alpha=alpha, test=test ) - for length in range(Lmax): + for length in range(max_history): for parent_id in range(len(states)): for history in sorted((h for h in states[parent_id] if len(h) == length), key=repr): for pair in pairs: @@ -270,16 +105,16 @@ def _edges( counts: JointSuffixCounts, alive: set[int], *, - Lmax: int, + max_history: int, alpha: float, - test: Literal["g", "chi2", "exact"], + test: TableTest, ) -> dict[int, dict[tuple[Any, Any], dict[int, set[JointHistory]]]]: """Successor states by ``(input, output)`` pair, with the histories that lead there. - Shorter histories move to their one-pair extension; length-``Lmax`` histories - drop their oldest pair, and the length-``Lmax + 1`` history is re-tested against + Shorter histories move to their one-pair extension; length-``max_history`` histories + drop their oldest pair, and the length-``max_history + 1`` history is re-tested against the truncated history's state (see - :func:`sofic.generators.epsilon_inference._suffix_edges`). + :func:`sofic.inference.cssr.process._suffix_edges`). """ in_alpha, out_alpha = counts.input_alphabet, counts.output_alphabet history_to_state = {h: index for index in alive for h in states[index]} @@ -292,7 +127,7 @@ def _edges( if not _history_emits(counts, history, pair): continue extended = (*history, pair) - if len(extended) <= Lmax: + if len(extended) <= max_history: target = history_to_state.get(extended) else: target = history_to_state.get(extended[1:]) @@ -330,36 +165,20 @@ def _edges( return edges -def _closed_classes(edges: dict[int, dict[tuple[Any, Any], dict[int, set[JointHistory]]]]) -> list[set[int]]: - import networkx as nx - - graph = nx.DiGraph() - graph.add_nodes_from(edges) - for source, by_pair in edges.items(): - for targets in by_pair.values(): - graph.add_edges_from((source, target) for target in targets) - condensed = nx.condensation(graph) - return [ - set(condensed.nodes[node]["members"]) - for node in condensed - if condensed.out_degree(node) == 0 and graph.subgraph(condensed.nodes[node]["members"]).number_of_edges() > 0 - ] - - def _determinize( states: list[set[JointHistory]], counts: JointSuffixCounts, alive: set[int], *, - Lmax: int, + max_history: int, alpha: float, - test: Literal["g", "chi2", "exact"], + test: TableTest, ) -> tuple[list[set[JointHistory]], set[int]]: """Split alive states until each ``(input, output)`` pair has one alive successor.""" states = [set(h) for h in states] alive = set(alive) while True: - edges = _edges(states, counts, alive, Lmax=Lmax, alpha=alpha, test=test) + edges = _edges(states, counts, alive, max_history=max_history, alpha=alpha, test=test) split = next( ( (index, pair) @@ -386,25 +205,25 @@ def _build_transducer( inputs: Sequence[Any], outputs: Sequence[Any], *, - Lmax: int, + max_history: int, alpha: float, - test: Literal["g", "chi2", "exact"], + test: TableTest, ) -> EpsilonTransducer: - edges = _edges(states, counts, alive, Lmax=Lmax, alpha=alpha, test=test) + edges = _edges(states, counts, alive, max_history=max_history, alpha=alpha, test=test) history_to_state = {h: index for index in alive for h in states[index]} visits: Counter[int] = Counter() pairs = tuple(zip(inputs, outputs, strict=True)) for t in range(len(pairs) + 1): - for hist_len in range(min(t, Lmax), -1, -1): + for hist_len in range(min(t, max_history), -1, -1): state = history_to_state.get(pairs[t - hist_len : t]) if state is not None: visits[state] += 1 break - classes = _closed_classes(edges) + classes = _recurrent_states(edges) if not classes: - raise StochasticValidationError("no recurrent inferred states; the sample is too short for this Lmax") + raise StochasticValidationError("no recurrent inferred states; the sample is too short for this max_history") keep = max(classes, key=lambda members: (sum(visits[s] for s in members), -min(members))) graph = TransitionGraph() @@ -452,7 +271,7 @@ def _build_transducer( return result -def _default_lmax(n: int, alphabet_size: int, min_count: int) -> int: +def _default_max_history(n: int, alphabet_size: int, min_count: int) -> int: if alphabet_size <= 0: return 1 # The joint (input, output) history space grows as ``alphabet_size ** L``, so @@ -460,15 +279,15 @@ def _default_lmax(n: int, alphabet_size: int, min_count: int) -> int: return max(1, min(5, n // max(1, alphabet_size * min_count))) -def transcssr( +def learn_epsilon_transducer_cssr( inputs: Sequence[Any], outputs: Sequence[Any], *, input_alphabet: Sequence[Any] | None = None, output_alphabet: Sequence[Any] | None = None, - Lmax: int | Literal["auto"] | None = None, + max_history: int | Literal["auto"] | None = None, alpha: float = 0.001, - test: Literal["g", "chi2", "exact"] = "g", + test: TableTest = "g", min_count: int = 5, correction: Literal["bonferroni"] | None = None, ) -> EpsilonTransducer: @@ -476,13 +295,13 @@ def transcssr( ``alpha`` is the per-test significance level for the causal-state split decision; the transCSSR/CSSR default of ``0.001`` favors fewer, more robust - states. ``Lmax`` bounds the joint-history depth and ``min_count`` the minimum + states. ``max_history`` bounds the joint-history depth and ``min_count`` the minimum occurrences before a history is eligible to seed a new state. - ``Lmax="auto"`` applies :func:`~sofic.generators.epsilon_inference.suggest_lmax` + ``max_history="auto"`` applies :func:`~sofic.inference.cssr.suggest_max_history` to the joint ``(input, output)`` sequence. ``test="exact"`` uses the Monte Carlo exact G-test when expected counts are small (see - :func:`~sofic.generators.epsilon_inference.morphs_differ`), and + :func:`~sofic.inference.cssr.morphs_differ`), and ``correction="bonferroni"`` divides ``alpha`` by the number of (history, input symbol) tests that can split a state. """ @@ -497,12 +316,12 @@ def transcssr( if input_alphabet is None or output_alphabet is None else len(tuple(input_alphabet)) * len(tuple(output_alphabet)) ) - if Lmax == "auto": - from sofic.generators.epsilon_inference import suggest_lmax - - max_length = suggest_lmax(list(zip(xs, ys, strict=True)), alpha=alpha) + if max_history == "auto": + max_length = suggest_max_history(list(zip(xs, ys, strict=True)), alpha=alpha) else: - max_length = Lmax if Lmax is not None else _default_lmax(len(xs), joint_alphabet_size, min_count) + max_length = ( + max_history if max_history is not None else _default_max_history(len(xs), joint_alphabet_size, min_count) + ) counts = JointSuffixCounts.from_sequences( xs, ys, @@ -520,9 +339,9 @@ def transcssr( elif correction is not None: raise ValueError(f"unknown correction {correction!r}") - states = _homogenize(counts, Lmax=max_length, alpha=alpha, test=test, min_count=min_count) + states = _homogenize(counts, max_history=max_length, alpha=alpha, test=test, min_count=min_count) everything = set(range(len(states))) - edges = _edges(states, counts, everything, Lmax=max_length, alpha=alpha, test=test) - alive = set().union(*_closed_classes(edges)) or everything - states, alive = _determinize(states, counts, alive, Lmax=max_length, alpha=alpha, test=test) - return _build_transducer(states, counts, alive, xs, ys, Lmax=max_length, alpha=alpha, test=test) + edges = _edges(states, counts, everything, max_history=max_length, alpha=alpha, test=test) + alive = set().union(*_recurrent_states(edges)) or everything + states, alive = _determinize(states, counts, alive, max_history=max_length, alpha=alpha, test=test) + return _build_transducer(states, counts, alive, xs, ys, max_history=max_length, alpha=alpha, test=test) diff --git a/sofic/inference/diagnostics.py b/sofic/inference/diagnostics.py index 9b97732..81a1d86 100644 --- a/sofic/inference/diagnostics.py +++ b/sofic/inference/diagnostics.py @@ -37,13 +37,13 @@ class GoodnessOfFit: ---------- statistic ``"g"`` or ``"entropy_rate"``. - L + block_length The word length compared. value The statistic on the observed data. pvalue ``(1 + #{simulated >= observed}) / (1 + n_samples)``. Small values mean - the machine does not reproduce the data's length-``L`` statistics. + the machine does not reproduce the data's length-``block_length`` statistics. null The statistic on each sequence simulated from the machine. forbidden_words @@ -51,7 +51,7 @@ class GoodnessOfFit: """ statistic: str - L: int + block_length: int value: float pvalue: float null: np.ndarray @@ -92,7 +92,7 @@ def goodness_of_fit( machine: Any, data: Sequence[Any], *, - L: int | None = None, + block_length: int | None = None, statistic: Literal["g", "entropy_rate"] = "g", n_samples: int = 199, burn_in: int = 100, @@ -101,8 +101,8 @@ def goodness_of_fit( """Parametric-bootstrap test that ``machine`` generated ``data``. Sequences as long as ``data`` are simulated from ``machine`` (after - ``burn_in`` steps, so they start near stationarity), and a length-``L`` word - statistic of the data is compared with its distribution over the simulations. + ``burn_in`` steps, so they start near stationarity), and a length-``block_length`` + word statistic of the data is compared with its distribution over the simulations. Because the null distribution is simulated, the overlap between successive words is accounted for; no chi-squared approximation is used. @@ -110,14 +110,14 @@ def goodness_of_fit( ---------- machine A fitted generator with ``sample``, ``word_probabilities`` and - ``stationary_distribution`` (e.g. an ε-machine from :func:`cssr`). + ``stationary_distribution`` (e.g. an ε-machine from :func:`learn_epsilon_machine_cssr`). data The observed sequence the machine was fitted to. - L + block_length Word length; by default the longest with about ten observations per possible word (between 1 and 6). statistic - ``"g"`` is the G statistic of the observed length-``L`` word counts + ``"g"`` is the G statistic of the observed length-``block_length`` word counts against the machine's stationary word probabilities. ``"entropy_rate"`` is ``|h_hat - h_L|``, the gap between the plug-in conditional entropy ``H[X_{L-1} | X_{0:L-1}]`` and the machine's value. @@ -137,7 +137,7 @@ def goodness_of_fit( Fitting and testing on the same data makes the test conservative, as in any parametric bootstrap without refitting. A small p-value is still evidence that the reconstruction misses structure. For CSSR that usually means - ``Lmax`` is shorter than the source's synchronization length, which happens + ``max_history`` is shorter than the source's synchronization length, which happens for strictly sofic sources. An observed word that the machine forbids gives ``G = inf`` and the smallest possible p-value. """ @@ -145,17 +145,17 @@ def goodness_of_fit( seq = tuple(data) n = len(seq) alphabet = set(seq) | set(machine.observation_alphabet) - if L is None: - L = _default_word_length(n, len(alphabet)) - if L < 1 or n < L: - raise ValueError("L must be between 1 and len(data)") + if block_length is None: + block_length = _default_word_length(n, len(alphabet)) + if block_length < 1 or n < block_length: + raise ValueError("block_length must be between 1 and len(data)") pi = np.asarray(machine.stationary_distribution(), dtype=float) - probabilities = {tuple(w): float(p) for w, p in machine.word_probabilities(L, start=pi).items()} + probabilities = {tuple(w): float(p) for w, p in machine.word_probabilities(block_length, start=pi).items()} if statistic == "g": def compute(sample: Sequence[Any]) -> float: - counts = _word_counts(sample, L) + counts = _word_counts(sample, block_length) total = sum(counts.values()) g = 0.0 for word, count in counts.items(): @@ -168,7 +168,7 @@ def compute(sample: Sequence[Any]) -> float: target = _model_conditional_entropy(probabilities) def compute(sample: Sequence[Any]) -> float: - return abs(_conditional_entropy(_word_counts(sample, L)) - target) + return abs(_conditional_entropy(_word_counts(sample, block_length)) - target) else: raise ValueError(f"unknown statistic {statistic!r}") @@ -179,8 +179,10 @@ def compute(sample: Sequence[Any]) -> float: null[i] = compute(simulated[burn_in:]) tol = 1e-12 * max(1.0, abs(value)) if np.isfinite(value) else 0.0 pvalue = float((1 + np.sum(null >= value - tol)) / (1 + n_samples)) - forbidden = tuple(sorted((w for w in _word_counts(seq, L) if probabilities.get(w, 0.0) <= 0.0), key=repr)) - return GoodnessOfFit(statistic, int(L), float(value), pvalue, null, forbidden) + forbidden = tuple( + sorted((w for w in _word_counts(seq, block_length) if probabilities.get(w, 0.0) <= 0.0), key=repr) + ) + return GoodnessOfFit(statistic, int(block_length), float(value), pvalue, null, forbidden) def topology_key(machine: Any) -> tuple[int, tuple[tuple[int, str, int], ...]]: @@ -301,7 +303,7 @@ def structure_stability( rng Seed or generator. **kwargs - Forwarded to the reconstruction (e.g. ``Lmax``, ``alpha``). + Forwarded to the reconstruction (e.g. ``max_history``, ``alpha``). Returns ------- @@ -351,24 +353,24 @@ def reconstruction_sweep( sequence: Sequence[Any], *, alphas: Sequence[float] = (0.05, 0.01, 0.001), - lmaxes: Sequence[int] = (1, 2, 3, 4), + max_histories: Sequence[int] = (1, 2, 3, 4), method: Literal["cssr"] | Callable[..., Any] = "cssr", **kwargs: Any, ) -> dict[tuple[float, int], tuple[int, tuple[tuple[int, str, int], ...]] | None]: """Reconstruct over a grid of significance levels and history lengths. - Returns ``{(alpha, Lmax): topology_key}``, with ``None`` where reconstruction - failed. A structure that persists across a range of ``alpha`` and ``Lmax`` is + Returns ``{(alpha, max_history): topology_key}``, with ``None`` where reconstruction + failed. A structure that persists across a range of ``alpha`` and ``max_history`` is better supported than one that appears at a single setting. For a Markov - source, it should be stable for every ``Lmax`` at or above the source's order. + source, it should be stable for every ``max_history`` at or above the source's order. """ results: dict[tuple[float, int], tuple[int, tuple[tuple[int, str, int], ...]] | None] = {} for alpha in alphas: - for lmax in lmaxes: + for history in max_histories: try: - machine = _reconstruct(sequence, method, {**kwargs, "alpha": alpha, "Lmax": lmax}) + machine = _reconstruct(sequence, method, {**kwargs, "alpha": alpha, "max_history": history}) except (StochasticValidationError, ValueError): - results[alpha, lmax] = None + results[alpha, history] = None continue - results[alpha, lmax] = topology_key(machine) + results[alpha, history] = topology_key(machine) return results diff --git a/sofic/inference/hmm/__init__.py b/sofic/inference/hmm/__init__.py new file mode 100644 index 0000000..04546f4 --- /dev/null +++ b/sofic/inference/hmm/__init__.py @@ -0,0 +1,37 @@ +"""Inference for hidden Markov models. + +Forward/backward/Viterbi decoding plus the Cappe, Moulines & Ryden (2005) +toolbox: fixed-interval smoothing (one- and two-slice marginals), Baum-Welch EM +parameter re-estimation, and the score / observed information via the Fisher +and Louis identities. +""" + +from sofic.inference.hmm.em import baum_welch +from sofic.inference.hmm.filtering import ( + backward, + forward, + log_likelihood, + smooth, + two_slice_marginals, + viterbi, +) +from sofic.inference.hmm.information import ( + free_parameter_labels, + observed_information, + score, + standard_errors, +) + +__all__ = [ + "backward", + "baum_welch", + "forward", + "free_parameter_labels", + "log_likelihood", + "observed_information", + "score", + "smooth", + "standard_errors", + "two_slice_marginals", + "viterbi", +] diff --git a/sofic/inference/hmm/em.py b/sofic/inference/hmm/em.py new file mode 100644 index 0000000..bc2860d --- /dev/null +++ b/sofic/inference/hmm/em.py @@ -0,0 +1,239 @@ +"""Baum-Welch (EM) parameter re-estimation for hidden Markov models. + +Follows Baum, Petrie, Soules & Weiss and Cappe, Moulines & Ryden (2005, Chapter 10). +""" + +from __future__ import annotations + +import warnings +from collections import defaultdict +from collections.abc import Iterable +from typing import Any + +import numpy as np + +from sofic.generators.base import HiddenMarkovModel +from sofic.generators.matrices import emission_tensors +from sofic.inference.hmm.filtering import _backward_scaled, _forward_scaled + + +def _expected_edge_counts( + pi: np.ndarray, + joint: dict[Any, np.ndarray], + obs: list[Any], +) -> tuple[dict[tuple[int, Any, int], float], np.ndarray, np.ndarray, float]: + r"""Expected sufficient statistics for one observation sequence. + + Returns ``(edge_counts, source_totals, gamma0, loglik)`` where + + - ``edge_counts[(i, symbol, j)]`` is + :math:`\sum_t P(X_t = i, Y_t = symbol, X_{t+1} = j \mid Y)`, the expected + number of uses of edge ``i --symbol--> j``; + - ``source_totals[i] = \sum_{t=0}^{n-1} P(X_t = i \mid Y)`` is the expected + number of transitions out of state ``i`` (the Baum-Welch denominator); + - ``gamma0`` is the smoothed marginal of the initial state ``X_0``; + - ``loglik`` is the log-likelihood of the sequence in bits. + + Only edges present in ``joint`` (structural support) receive mass, so the + statistics preserve the model topology. + """ + n_states = len(pi) + alpha_hat, log_scales = _forward_scaled(pi, joint, obs) + if not np.all(np.isfinite(log_scales)): + return {}, np.zeros(n_states), np.zeros(n_states), float("-inf") + beta_hat = _backward_scaled(joint, obs, n_states) + edge_counts: dict[tuple[int, Any, int], float] = {} + source_totals = np.zeros(n_states, dtype=float) + for t, symbol in enumerate(obs): + matrix = joint.get(symbol) + if matrix is None: + continue + block = alpha_hat[t][:, None] * matrix * beta_hat[t + 1][None, :] + total = float(block.sum()) + if total <= 0.0: + continue + block = block / total + source_totals += block.sum(axis=1) + for i, j in np.argwhere(block > 0.0): + key = (int(i), symbol, int(j)) + edge_counts[key] = edge_counts.get(key, 0.0) + float(block[i, j]) + g0 = alpha_hat[0] * beta_hat[0] + s0 = float(g0.sum()) + gamma0 = g0 / s0 if s0 > 0.0 else np.zeros(n_states) + return edge_counts, source_totals, gamma0, float(log_scales.sum()) + + +def _as_sequence_list(sequences: Iterable[Any]) -> list[list[Any]]: + """Normalize ``sequences`` to a list of observation sequences. + + Accepts either a single flat observation sequence (e.g. ``[0, 1, 0]``) or an + iterable of sequences (e.g. ``[[0, 1], [1, 0]]``). A single sequence is + detected when its first element is not itself a non-string sequence. + """ + seqs = list(sequences) + if not seqs: + return [] + first = seqs[0] + if isinstance(first, (list, tuple)) and not isinstance(first, (str, bytes)): + return [list(seq) for seq in seqs] + return [seqs] + + +def baum_welch( + hmm: HiddenMarkovModel, + sequences: Iterable[Any], + *, + max_iter: int = 100, + tol: float = 1e-6, + estimate_initial: bool = True, + n_restarts: int = 1, + rng: np.random.Generator | int | None = None, + return_restarts: bool = False, +) -> tuple[Any, list[float]] | tuple[Any, list[float], list[float]]: + r"""Fit HMM parameters by Baum-Welch (EM) expectation-maximization. + + Re-estimates the Mealy joint edge law + :math:`A_o[i, j] = P(X_{t+1} = j, O = o \mid X_t = i)` and (optionally) the + initial distribution from data, holding the transition-graph topology fixed: + structurally absent edges receive zero expected count and stay absent, so the + fitted model generates the same sofic shift as ``hmm``. This is the EM + algorithm for probabilistic functions of finite Markov chains of Baum, Petrie, + Soules & Weiss and Cappe, Moulines & Ryden (2005, Chapter 10); see also + Rabiner (1989). + + ``sequences`` may be a single observation sequence or an iterable of + sequences (several sequences are needed to identify the initial distribution; + Cappe, Moulines & Ryden, 2005, Section 10.1). Unifilarity is *not* preserved, + so the fit is returned as a plain :class:`~sofic.generators.mealy.MealyHMM`. + + Returns ``(fitted_model, loglik_trace)`` where ``loglik_trace`` is the + non-decreasing sequence of total log-likelihoods (bits) observed before each + parameter update. + + EM converges to a local maximum of the likelihood. With ``n_restarts > 1`` the + first run starts from ``hmm``'s parameters and each further run from edge laws + drawn uniformly (Dirichlet(1)) over each state's structurally allowed edges; + the fit with the highest final log-likelihood is returned. Pass + ``return_restarts=True`` to also get every run's final log-likelihood, which + shows whether near-equal optima exist. + """ + from sofic.generators.mealy import MealyHMM + + mealy = hmm.to_mealy() + idx = mealy.reindex() + n_states = len(idx) + states = [idx.state(i) for i in range(n_states)] + alphabet = frozenset(mealy.observation_alphabet) + seqs = _as_sequence_list(sequences) + + pi, joint = emission_tensors(mealy) + support = { + (i, symbol, j) + for symbol, matrix in joint.items() + for i in range(n_states) + for j in range(n_states) + if matrix[i, j] > 0.0 + } + + if n_restarts < 1: + raise ValueError("n_restarts must be at least 1") + generator = rng if isinstance(rng, np.random.Generator) else np.random.default_rng(rng) + runs = [] + for restart in range(n_restarts): + start_joint = joint if restart == 0 else _random_edge_law(joint, support, n_states, generator) + runs.append( + _baum_welch_run(pi, start_joint, seqs, max_iter=max_iter, tol=tol, estimate_initial=estimate_initial) + ) + finals = [trace[-1] if trace else float("-inf") for _pi, _joint, trace in runs] + pi, joint, loglik_trace = runs[int(np.argmax(finals))] + + fitted = MealyHMM( + initial_distribution={states[i]: float(pi[i]) for i in range(n_states) if pi[i] > 0.0}, + observation_alphabet=alphabet, + ) + for state in states: + fitted.graph.add_state(state) + for i, symbol, j in sorted(support, key=lambda edge: (edge[0], str(edge[1]), edge[2])): + prob = float(joint[symbol][i, j]) + if prob > 0.0: + fitted.add_transition(states[i], states[j], symbol, prob) + fitted.validate() + if return_restarts: + return fitted, loglik_trace, finals + return fitted, loglik_trace + + +def _random_edge_law( + joint: dict[Any, np.ndarray], + support: set[tuple[int, Any, int]], + n_states: int, + rng: np.random.Generator, +) -> dict[Any, np.ndarray]: + """Edge laws drawn uniformly over each state's allowed ``(symbol, target)`` edges.""" + new_joint = {symbol: np.zeros((n_states, n_states), dtype=float) for symbol in joint} + for i in range(n_states): + edges = sorted(((symbol, j) for (source, symbol, j) in support if source == i), key=lambda e: (str(e[0]), e[1])) + if not edges: + continue + weights = rng.dirichlet(np.ones(len(edges))) + for (symbol, j), weight in zip(edges, weights, strict=True): + new_joint[symbol][i, j] = weight + return new_joint + + +def _baum_welch_run( + pi: np.ndarray, + joint: dict[Any, np.ndarray], + seqs: list[Any], + *, + max_iter: int, + tol: float, + estimate_initial: bool, +) -> tuple[np.ndarray, dict[Any, np.ndarray], list[float]]: + """One EM run from ``(pi, joint)``; returns the final parameters and trace.""" + n_states = len(pi) + loglik_trace: list[float] = [] + prev_ll: float | None = None + for _iteration in range(max_iter): + total_edge_counts: dict[tuple[int, Any, int], float] = defaultdict(float) + total_source = np.zeros(n_states, dtype=float) + gamma0_sum = np.zeros(n_states, dtype=float) + total_ll = 0.0 + skipped = 0 + for obs in seqs: + edge_counts, source_totals, gamma0, loglik = _expected_edge_counts(pi, joint, obs) + if not np.isfinite(loglik): + skipped += 1 + continue + for key, value in edge_counts.items(): + total_edge_counts[key] += value + total_source += source_totals + gamma0_sum += gamma0 + total_ll += loglik + if seqs and skipped == len(seqs): + raise ValueError("every observation sequence has zero probability under the model") + if skipped and _iteration == 0: + warnings.warn( + f"{skipped} of {len(seqs)} sequences have zero probability under the model and are ignored", + RuntimeWarning, + stacklevel=3, + ) + loglik_trace.append(total_ll) + if prev_ll is not None and abs(total_ll - prev_ll) < tol: + break + prev_ll = total_ll + + new_joint = {symbol: np.zeros((n_states, n_states), dtype=float) for symbol in joint} + for (i, symbol, j), count in total_edge_counts.items(): + if total_source[i] > 0.0: + new_joint[symbol][i, j] = count / total_source[i] + for i in range(n_states): + if total_source[i] <= 0.0: + for symbol in joint: + new_joint[symbol][i, :] = joint[symbol][i, :] + joint = new_joint + if estimate_initial: + mass = float(gamma0_sum.sum()) + if mass > 0.0: + pi = gamma0_sum / mass + return pi, joint, loglik_trace diff --git a/sofic/inference/hmm/filtering.py b/sofic/inference/hmm/filtering.py new file mode 100644 index 0000000..59ce68d --- /dev/null +++ b/sofic/inference/hmm/filtering.py @@ -0,0 +1,237 @@ +"""Forward/backward filtering, smoothing, and Viterbi decoding for hidden Markov models. + +Fixed-interval smoothing (one- and two-slice marginals) follows Cappe, Moulines & +Ryden (2005, Section 3.2). +""" + +from __future__ import annotations + +from collections.abc import Hashable, Sequence +from typing import Any + +import numpy as np + +from sofic.generators.base import HiddenMarkovModel +from sofic.generators.matrices import emission_tensors, symbol_matrices + + +def _as_mealy_hmm(hmm: HiddenMarkovModel) -> Any: + """Return a Mealy-style representation through the HMM representation hook.""" + return hmm.to_mealy() + + +def _forward_scaled( + pi: np.ndarray, + joint: dict[Any, np.ndarray], + obs: list[Any], +) -> tuple[np.ndarray, np.ndarray]: + """Return per-step-normalized forward messages and log scaling factors. + + ``alpha_hat[t]`` sums to one; ``log2 P(obs) = log_scales.sum()``. A ``-inf`` + entry in ``log_scales`` marks an impossible step. Normalizing each step avoids + the underflow that makes the raw forward product vanish for long sequences. + """ + n = len(pi) + alpha_hat = np.zeros((len(obs) + 1, n), dtype=float) + log_scales = np.zeros(len(obs) + 1, dtype=float) + total0 = float(pi.sum()) + if total0 <= 0.0: + log_scales[0] = -np.inf + return alpha_hat, log_scales + alpha_hat[0] = pi / total0 + log_scales[0] = float(np.log2(total0)) + for t, symbol in enumerate(obs): + matrix = joint.get(symbol) + if matrix is None: + log_scales[t + 1] = -np.inf + continue + row = alpha_hat[t] @ matrix + scale = float(row.sum()) + if scale <= 0.0: + log_scales[t + 1] = -np.inf + continue + alpha_hat[t + 1] = row / scale + log_scales[t + 1] = float(np.log2(scale)) + return alpha_hat, log_scales + + +def forward(hmm: HiddenMarkovModel, observations: Sequence[Any], *, normalize: bool = False) -> np.ndarray: + """Return forward messages ``alpha[t, s]`` for ``len(observations)+1`` rows. + + With ``normalize=True`` each row is normalized to sum to one (the numerically + stable message used for posteriors); otherwise the raw messages are returned. + """ + pi, joint = emission_tensors(hmm) + obs = list(observations) + if normalize: + alpha_hat, _log_scales = _forward_scaled(pi, joint, obs) + return alpha_hat + n = len(pi) + alpha = np.zeros((len(obs) + 1, n), dtype=float) + alpha[0] = pi + for t, symbol in enumerate(obs): + matrix = joint.get(symbol) + if matrix is None: + alpha[t + 1] = 0.0 + else: + alpha[t + 1] = alpha[t] @ matrix + return alpha + + +def backward(hmm: HiddenMarkovModel, observations: Sequence[Any], *, normalize: bool = False) -> np.ndarray: + """Return backward messages ``beta[t, s]`` for ``len(observations)+1`` rows. + + With ``normalize=True`` each row is normalized to sum to one. The smoothed + posterior is then ``normalize(alpha_hat[t] * beta_hat[t])`` (the per-row + scaling constants cancel on renormalization). + """ + joint = symbol_matrices(_as_mealy_hmm(hmm)) + n = next(iter(joint.values())).shape[0] if joint else len(_as_mealy_hmm(hmm).reindex()) + obs = list(observations) + beta = np.zeros((len(obs) + 1, n), dtype=float) + beta[len(obs)] = 1.0 + for t in range(len(obs) - 1, -1, -1): + matrix = joint.get(obs[t]) + if matrix is None: + beta[t] = 0.0 + else: + beta[t] = matrix @ beta[t + 1] + if normalize: + total = float(beta[t].sum()) + if total > 0.0: + beta[t] = beta[t] / total + return beta + + +def _backward_scaled(joint: dict[Any, np.ndarray], obs: list[Any], n_states: int) -> np.ndarray: + """Per-row-normalized backward messages from precomputed transition tensors. + + ``beta_hat[t]`` sums to one; the per-row scaling constants cancel against the + forward scaling when the smoothed posterior is renormalized. Shares tensors + with the forward pass so smoothing and EM avoid recomputing them. + """ + beta = np.zeros((len(obs) + 1, n_states), dtype=float) + beta[len(obs)] = 1.0 + for t in range(len(obs) - 1, -1, -1): + matrix = joint.get(obs[t]) + row = beta[t + 1] if matrix is None else matrix @ beta[t + 1] + beta[t] = 0.0 if matrix is None else row + total = float(beta[t].sum()) + if total > 0.0: + beta[t] = beta[t] / total + return beta + + +def log_likelihood(hmm: HiddenMarkovModel, observations: Sequence[Any]) -> float: + """Log-likelihood ``log2 P(observations)`` in bits. + + Uses the per-step-scaled forward recursion so the result stays finite for long + sequences instead of underflowing to ``-inf``. + """ + pi, joint = emission_tensors(hmm) + _alpha_hat, log_scales = _forward_scaled(pi, joint, list(observations)) + if not np.all(np.isfinite(log_scales)): + return float("-inf") + return float(log_scales.sum()) + + +def smooth(hmm: HiddenMarkovModel, observations: Sequence[Any]) -> np.ndarray: + r"""Return fixed-interval smoothed marginals ``gamma[t, s]``. + + ``gamma[t, s] = P(X_t = s \mid Y_{0:n-1})`` for ``t = 0, ..., n`` (there are + ``n + 1`` hidden states behind ``n`` edge emissions). Computed as the + per-row-renormalized product of the scaled forward and backward messages, the + forward-backward smoother of Cappe, Moulines & Ryden (2005, Section 3.2). + Rows for observation sequences of zero probability are returned as zeros. + """ + pi, joint = emission_tensors(hmm) + obs = list(observations) + n_states = len(pi) + alpha_hat, log_scales = _forward_scaled(pi, joint, obs) + if not np.all(np.isfinite(log_scales)): + return np.zeros((len(obs) + 1, n_states), dtype=float) + beta_hat = _backward_scaled(joint, obs, n_states) + gamma = alpha_hat * beta_hat + row_sums = gamma.sum(axis=1, keepdims=True) + with np.errstate(invalid="ignore", divide="ignore"): + gamma = np.where(row_sums > 0.0, gamma / row_sums, 0.0) + return gamma + + +def two_slice_marginals(hmm: HiddenMarkovModel, observations: Sequence[Any]) -> np.ndarray: + r"""Return two-slice smoothed marginals ``xi[t, i, j]``. + + ``xi[t, i, j] = P(X_t = i, X_{t+1} = j \mid Y_{0:n-1})`` for ``t = 0, ..., n-1``, + where the transition at index ``t`` emits ``Y_t`` (Cappe, Moulines & Ryden, + 2005, Section 3.2). Marginalizing over ``j`` recovers ``gamma[t]`` for + ``t < n``. Returns an all-zero tensor for zero-probability sequences. + """ + pi, joint = emission_tensors(hmm) + obs = list(observations) + n_states = len(pi) + xi = np.zeros((len(obs), n_states, n_states), dtype=float) + alpha_hat, log_scales = _forward_scaled(pi, joint, obs) + if not np.all(np.isfinite(log_scales)): + return xi + beta_hat = _backward_scaled(joint, obs, n_states) + for t, symbol in enumerate(obs): + matrix = joint.get(symbol) + if matrix is None: + continue + block = alpha_hat[t][:, None] * matrix * beta_hat[t + 1][None, :] + total = float(block.sum()) + if total > 0.0: + xi[t] = block / total + return xi + + +def _log_probabilities(values: np.ndarray) -> np.ndarray: + log_values = np.full(values.shape, -np.inf, dtype=float) + positive = values > 0.0 + log_values[positive] = np.log(values[positive]) + return log_values + + +def viterbi(hmm: HiddenMarkovModel, observations: Sequence[Any]) -> list[Hashable]: + mealy = _as_mealy_hmm(hmm) + idx = mealy.reindex() + pi, joint = emission_tensors(mealy) + n = len(idx) + obs = list(observations) + if n == 0: + return [] + if not obs: + if not np.any(pi > 0.0): + return [] + return [idx.state(int(np.argmax(pi)))] + + log_pi = _log_probabilities(pi) + viterbi_log = np.full((len(obs), n), -np.inf, dtype=float) + backpointer = np.full((len(obs), n), -1, dtype=int) + + matrix0 = joint.get(obs[0]) + if matrix0 is not None: + log_matrix0 = _log_probabilities(matrix0) + for j in range(n): + best = log_pi + log_matrix0[:, j] + viterbi_log[0, j] = np.max(best) + backpointer[0, j] = int(np.argmax(best)) + + for t in range(1, len(obs)): + matrix = joint.get(obs[t]) + if matrix is None: + continue + log_matrix = _log_probabilities(matrix) + for j in range(n): + scores = viterbi_log[t - 1] + log_matrix[:, j] + viterbi_log[t, j] = np.max(scores) + backpointer[t, j] = int(np.argmax(scores)) + + if not np.any(np.isfinite(viterbi_log[-1])): + return [] + + path = [0] * len(obs) + path[-1] = int(np.argmax(viterbi_log[-1])) + for t in range(len(obs) - 2, -1, -1): + path[t] = backpointer[t + 1, path[t + 1]] + return [idx.state(i) for i in path] diff --git a/sofic/inference/hmm/information.py b/sofic/inference/hmm/information.py new file mode 100644 index 0000000..66b04f9 --- /dev/null +++ b/sofic/inference/hmm/information.py @@ -0,0 +1,208 @@ +"""Score and observed information of hidden Markov models. + +The score follows from the Fisher identity and the observed information from +Louis' identity (Cappe, Moulines & Ryden, 2005, Section 10.2.3). +""" + +from __future__ import annotations + +from collections.abc import Hashable, Sequence +from typing import Any + +import numpy as np + +from sofic.generators.base import HiddenMarkovModel +from sofic.generators.matrices import emission_tensors, symbol_matrices +from sofic.inference.hmm.em import _expected_edge_counts +from sofic.inference.hmm.filtering import _forward_scaled + + +def score(hmm: HiddenMarkovModel, observations: Sequence[Any]) -> dict[tuple[Hashable, Any, Hashable], float]: + r"""Return the score (gradient of the log-likelihood) via the Fisher identity. + + For each edge ``i --o--> j``, returns + :math:`\partial \log P(Y) / \partial A_o[i, j] = E[N_{i,o,j} \mid Y] / A_o[i, j]`, + where ``N`` is the (unobserved) edge-use count. This is Fisher's identity, + ``\nabla \log L(\theta) = E[\nabla \log f(X, Y; \theta) \mid Y]`` (Cappe, + Moulines & Ryden, 2005, Section 10.2.3), evaluated in the raw (unconstrained) + joint-edge parameters. Keys are ``(source, symbol, target)`` state labels. + + The score is the gradient of the *natural* log-likelihood + :math:`\ln P(Y) = \ln 2 \cdot` :func:`log_likelihood`, the convention under + which the observed information and standard errors are standard. + """ + mealy = hmm.to_mealy() + idx = mealy.reindex() + pi, joint = emission_tensors(mealy) + edge_counts, _source_totals, _gamma0, loglik = _expected_edge_counts(pi, joint, list(observations)) + if not np.isfinite(loglik): + raise ValueError("observations have zero probability under the model; score is undefined") + result: dict[tuple[Hashable, Any, Hashable], float] = {} + n_states = len(pi) + for symbol, matrix in joint.items(): + for i in range(n_states): + for j in range(n_states): + prob = float(matrix[i, j]) + if prob > 0.0: + count = edge_counts.get((i, symbol, j), 0.0) + result[(idx.state(i), symbol, idx.state(j))] = count / prob + return result + + +def _free_parameterization( + joint: dict[Any, np.ndarray], + n_states: int, +) -> tuple[list[tuple[int, Any, int]], list[tuple[int, Any, int]], list[int]]: + """Build the free multinomial parameterization of the joint edge law. + + Each source state whose outgoing edges number ``k >= 2`` contributes ``k - 1`` + free parameters (its last edge in canonical order is the reference). Returns + ``(free_edges, reference_by_param, source_by_param)``: the edge for each free + parameter, the reference edge of its source block, and the source-state index. + """ + free_edges: list[tuple[int, Any, int]] = [] + reference_by_param: list[tuple[int, Any, int]] = [] + source_by_param: list[int] = [] + for i in range(n_states): + out_edges = sorted( + ((i, symbol, j) for symbol, matrix in joint.items() for j in range(n_states) if matrix[i, j] > 0.0), + key=lambda edge: (str(edge[1]), edge[2]), + ) + if len(out_edges) < 2: + continue + reference = out_edges[-1] + for edge in out_edges[:-1]: + free_edges.append(edge) + reference_by_param.append(reference) + source_by_param.append(i) + return free_edges, reference_by_param, source_by_param + + +def observed_information(hmm: HiddenMarkovModel, observations: Sequence[Any]) -> np.ndarray: + r"""Return the observed information matrix via Louis' identity. + + The observed information ``J = -\partial^2 \log L / \partial\theta^2`` for the + free multinomial parameters of the joint edge law is obtained from Louis' + (1982) identity, + + .. math:: J = E[-\partial^2 \ell_c \mid Y] - \operatorname{Cov}(\partial \ell_c \mid Y), + + where :math:`\ell_c` is the complete-data log-likelihood (Cappe, Moulines & + Ryden, 2005, Section 10.2.3). The complete-data information ``B`` follows from + the expected edge counts; the conditional covariance of the complete-data + score is computed exactly by a forward smoothing recursion for the first and + second moments of the additive score functional. The matrix is ordered by + :func:`free_parameter_labels`; an empty ``(0, 0)`` matrix is returned when the + model has no free parameters. + """ + mealy = hmm.to_mealy() + pi, joint = emission_tensors(mealy) + obs = list(observations) + n_states = len(pi) + + free_edges, reference_by_param, source_by_param = _free_parameterization(joint, n_states) + d = len(free_edges) + if d == 0: + return np.zeros((0, 0), dtype=float) + + edge_counts, _source_totals, _gamma0, loglik = _expected_edge_counts(pi, joint, obs) + if not np.isfinite(loglik): + raise ValueError("observations have zero probability under the model; information is undefined") + + prob_of = {edge: float(joint[edge[1]][edge[0], edge[2]]) for edge in set(free_edges) | set(reference_by_param)} + + # Complete-data information B = E[-d^2 l_c | Y], block-diagonal by source state. + complete_information = np.zeros((d, d), dtype=float) + for p in range(d): + ref_p = reference_by_param[p] + count_ref = edge_counts.get(ref_p, 0.0) + ref_term = count_ref / prob_of[ref_p] ** 2 + for q in range(d): + if source_by_param[p] != source_by_param[q]: + continue + value = ref_term + if p == q: + edge_p = free_edges[p] + value += edge_counts.get(edge_p, 0.0) / prob_of[edge_p] ** 2 + complete_information[p, q] = value + + # Per-transition score contribution s(edge) as a d-vector (sparse per source block). + edge_score: dict[tuple[int, Any, int], np.ndarray] = {} + for p, edge in enumerate(free_edges): + edge_score.setdefault(edge, np.zeros(d))[p] += 1.0 / prob_of[edge] + for p, ref in enumerate(reference_by_param): + edge_score.setdefault(ref, np.zeros(d))[p] += -1.0 / prob_of[ref] + zero_d = np.zeros(d) + + # Forward smoothing recursion for E[S | Y] and E[S S^T | Y] of the additive + # complete-data score functional S = sum_t s(edge_t). + alpha_hat, _log_scales = _forward_scaled(pi, joint, obs) + first = np.zeros((n_states, d), dtype=float) + second = np.zeros((n_states, d, d), dtype=float) + for t, symbol in enumerate(obs): + matrix = joint.get(symbol) + if matrix is None: + continue + weight = alpha_hat[t][:, None] * matrix # weight[i, k] = P(X_t=i, X_{t+1}=k, Y_t | Y_{0:t-1}) + denom = weight.sum(axis=0) + new_first = np.zeros((n_states, d), dtype=float) + new_second = np.zeros((n_states, d, d), dtype=float) + for k in range(n_states): + if denom[k] <= 0.0: + continue + for i in range(n_states): + if weight[i, k] <= 0.0: + continue + retro = weight[i, k] / denom[k] # P(X_t=i | X_{t+1}=k, Y_{0:t}) + s_vec = edge_score.get((i, symbol, k), zero_d) + first_i = first[i] + combined = first_i + s_vec + new_first[k] += retro * combined + cross = np.outer(first_i, s_vec) + new_second[k] += retro * (second[i] + cross + cross.T + np.outer(s_vec, s_vec)) + first, second = new_first, new_second + + phi_final = alpha_hat[len(obs)] + expected_score = phi_final @ first + expected_outer = np.einsum("k,kpq->pq", phi_final, second) + score_covariance = expected_outer - np.outer(expected_score, expected_score) + return complete_information - score_covariance + + +def free_parameter_labels(hmm: HiddenMarkovModel) -> list[tuple[Hashable, Any, Hashable]]: + """Return the ``(source, symbol, target)`` label for each free parameter. + + The order matches the rows and columns of :func:`observed_information` and the + entries of :func:`standard_errors`. + """ + mealy = hmm.to_mealy() + idx = mealy.reindex() + joint = symbol_matrices(mealy) + free_edges, _reference, _source = _free_parameterization(joint, len(idx)) + return [(idx.state(i), symbol, idx.state(j)) for i, symbol, j in free_edges] + + +def standard_errors( + hmm: HiddenMarkovModel, + observations: Sequence[Any], +) -> dict[tuple[Hashable, Any, Hashable], float]: + r"""Return asymptotic standard errors of the free edge parameters. + + Standard errors are ``sqrt(diag(J^{-1}))`` where ``J`` is the + :func:`observed_information` matrix (Cappe, Moulines & Ryden, 2005, + Section 10.2.3). Uses the Moore-Penrose pseudoinverse when ``J`` is singular; + a non-positive variance estimate (numerically unidentified parameter) yields + ``nan``. Keyed by the labels from :func:`free_parameter_labels`. + """ + labels = free_parameter_labels(hmm) + information = observed_information(hmm, observations) + if information.shape[0] == 0: + return {} + try: + covariance = np.linalg.inv(information) + except np.linalg.LinAlgError: + covariance = np.linalg.pinv(information) + variances = np.diag(covariance) + with np.errstate(invalid="ignore"): + errors = np.where(variances > 0.0, np.sqrt(variances), np.nan) + return dict(zip(labels, (float(value) for value in errors), strict=True)) diff --git a/sofic/inference/model_selection.py b/sofic/inference/model_selection.py index c108140..d48dc9a 100644 --- a/sofic/inference/model_selection.py +++ b/sofic/inference/model_selection.py @@ -1,4 +1,4 @@ -"""Classical model-selection criteria for stochastic generators. +r"""Classical model-selection criteria for stochastic generators. Point-estimate information criteria -- AIC :cite:`Akaike1974`, the small-sample-corrected AICc :cite:`HurvichTsai1989`, BIC :cite:`Schwarz1978`, @@ -7,11 +7,15 @@ criterion (WAIC) :cite:`Watanabe2010`. These complement the exact Bayesian evidences of :mod:`sofic.inference.bayesian`: they score any fitted :class:`~sofic.generators.base.HiddenMarkovModel` (ε-machine, Mealy HMM, Markov -chain) using the natural-log likelihood from -:func:`sofic.generators.hmm_inference.log_likelihood` and a free-parameter count +chain) using the log-likelihood (in bits) from +:func:`sofic.inference.hmm.log_likelihood` and a free-parameter count read off the transition graph, so they are likelihood-agnostic and apply directly to discrete-emission models. +Log-likelihoods, log scores, and the MDL code length are reported in **bits**. +AIC, AICc, BIC, and WAIC keep their standard deviance scale: they are computed +from the natural log-likelihood :math:`\ln L = \ln 2 \cdot \log_2 L`. + All information criteria follow the convention **lower is better**; cross-validated and WAIC log scores follow **higher is better** for the raw log score (WAIC itself is reported on the deviance scale, lower is better). @@ -26,7 +30,7 @@ import numpy as np from sofic.generators.base import HiddenMarkovModel -from sofic.generators.hmm_inference import free_parameter_labels, log_likelihood +from sofic.inference.hmm import free_parameter_labels, log_likelihood __all__ = [ "ModelScores", @@ -90,7 +94,7 @@ def count_free_parameters(model: HiddenMarkovModel, *, include_initial: bool = F Counts one free parameter per non-reference outgoing edge at each state (the multinomial free-parameterization of the joint emission-transition law used - by :func:`sofic.generators.hmm_inference.observed_information`). With + by :func:`sofic.inference.hmm.observed_information`). With ``include_initial`` the ``n - 1`` free parameters of the initial distribution are added; for a stationary presentation the initial law is determined by the dynamics, so this defaults to ``False``. @@ -123,15 +127,15 @@ def _smoothed_log_likelihood( smoothing: float, alphabet_size: int, ) -> float: - """Natural-log likelihood with each one-step prediction mixed with the uniform law. + """Log-likelihood (bits) with each one-step prediction mixed with the uniform law. ``P'(x_t | x_{0:t}) = (1 - smoothing) P(x_t | x_{0:t}) + smoothing / alphabet_size``, computed by forward filtering. After a symbol the model forbids, the belief is propagated without conditioning on it. """ - from sofic.generators.hmm_inference import _emission_transition_tensors_from_mealy + from sofic.generators.matrices import emission_tensors - pi, joint = _emission_transition_tensors_from_mealy(model.to_mealy()) + pi, joint = emission_tensors(model) total_step = sum(joint.values()) belief = np.asarray(pi, dtype=float) belief = belief / belief.sum() @@ -140,7 +144,7 @@ def _smoothed_log_likelihood( matrix = joint.get(symbol) unnormalized = belief @ matrix if matrix is not None else np.zeros_like(belief) predicted = float(unnormalized.sum()) - total += float(np.log((1.0 - smoothing) * predicted + smoothing / alphabet_size)) + total += float(np.log2((1.0 - smoothing) * predicted + smoothing / alphabet_size)) if predicted > 0.0: belief = unnormalized / predicted else: @@ -158,8 +162,9 @@ def score_model( """Score ``model`` on ``data`` with AIC, AICc, BIC, and MDL. ``data`` may be a single observation sequence or an iterable of sequences. - The scores use natural-log likelihoods; the number of observations is the - total symbol count. When the data has zero probability under the model the + ``log_likelihood`` and ``mdl`` are in bits; AIC, AICc, and BIC use the natural + log-likelihood so they keep their usual scale. The number of observations is + the total symbol count. When the data has zero probability under the model the likelihood is ``-inf`` and every criterion is ``+inf``. """ sequences = _normalize_sequences(data) @@ -171,12 +176,12 @@ def score_model( inf = float("inf") return ModelScores(float("-inf"), k, n, inf, inf, inf, inf) - aic = 2.0 * k - 2.0 * ll + ln_l = ll * np.log(2.0) + aic = 2.0 * k - 2.0 * ln_l denom = n - k - 1 aicc = aic + (2.0 * k * (k + 1)) / denom if denom > 0 else float("inf") - log_n = np.log(n) if n > 0 else 0.0 - bic = k * log_n - 2.0 * ll - mdl = 0.5 * k * log_n - ll + bic = k * (np.log(n) if n > 0 else 0.0) - 2.0 * ln_l + mdl = 0.5 * k * (np.log2(n) if n > 0 else 0.0) - ll return ModelScores(float(ll), k, n, float(aic), float(aicc), float(bic), float(mdl)) @@ -229,7 +234,7 @@ def cross_validated_log_likelihood( gap: int = 0, smoothing: float = 0.0, ) -> float: - """Return the total held-out natural-log likelihood under ``folds``-fold CV. + """Return the total held-out log-likelihood (bits) under ``folds``-fold CV. ``fit(train_sequences)`` must fit and return a model from a list of training sequences. When ``data`` is a collection of sequences the folds partition the @@ -249,7 +254,7 @@ def cross_validated_log_likelihood( smoothing Mix each one-step held-out prediction with the uniform distribution over the observed alphabet, with this weight. Then a single transition that the - fitted model forbids costs ``log(smoothing / |A|)`` instead of making the + fitted model forbids costs ``log2(smoothing / |A|)`` instead of making the whole fold ``-inf``, so models can still be compared. """ if gap < 0: @@ -307,17 +312,20 @@ def waic(pointwise_log_likelihoods: np.ndarray) -> WAICResult: r"""Widely applicable information criterion from posterior samples. ``pointwise_log_likelihoods`` has shape ``(n_samples, n_points)`` with entry - ``[s, i] = log p(y_i | theta_s)`` for posterior draw ``theta_s``. Returns the - WAIC on the deviance scale (lower is better), + ``[s, i] = log2 p(y_i | theta_s)`` (bits) for posterior draw ``theta_s``, as + returned by :func:`posterior_pointwise_log_likelihoods`. Returns the WAIC on + the standard (natural-log) deviance scale (lower is better), ``WAIC = -2 (lppd - p_waic)`` with the log pointwise predictive density ``lppd = sum_i log mean_s p(y_i | theta_s)`` and effective parameter count - ``p_waic = sum_i Var_s log p(y_i | theta_s)`` :cite:`Watanabe2010`. + ``p_waic = sum_i Var_s log p(y_i | theta_s)`` :cite:`Watanabe2010`. ``waic``, + ``p_waic``, and ``standard_error`` use natural logs; ``lppd`` is in bits. """ from scipy.special import logsumexp matrix = np.asarray(pointwise_log_likelihoods, dtype=float) if matrix.ndim != 2 or matrix.size == 0: raise ValueError("pointwise_log_likelihoods must be a non-empty (n_samples, n_points) array") + matrix = matrix * np.log(2.0) n_samples = matrix.shape[0] lppd_pointwise = logsumexp(matrix, axis=0) - np.log(n_samples) p_waic_pointwise = matrix.var(axis=0, ddof=1) if n_samples > 1 else np.zeros(matrix.shape[1]) @@ -327,7 +335,7 @@ def waic(pointwise_log_likelihoods: np.ndarray) -> WAICResult: standard_error = float(np.sqrt(n_points * np.var(-2.0 * elpd_pointwise, ddof=0))) if n_points > 1 else 0.0 return WAICResult( waic=waic_value, - lppd=float(lppd_pointwise.sum()), + lppd=float(lppd_pointwise.sum() / np.log(2.0)), p_waic=float(p_waic_pointwise.sum()), standard_error=standard_error, ) diff --git a/sofic/inference/spectral.py b/sofic/inference/spectral.py index abf9b23..d24d454 100644 --- a/sofic/inference/spectral.py +++ b/sofic/inference/spectral.py @@ -33,12 +33,15 @@ from collections import defaultdict from collections.abc import Callable, Hashable, Iterable, Sequence -from typing import Any +from typing import TYPE_CHECKING, Any import numpy as np from sofic.generators.quasi_realization import QuasiRealization +if TYPE_CHECKING: + from sofic.generators.epsilon_machine import EpsilonMachine + __all__ = [ "SpectralInferenceError", "hankel_matrices", @@ -46,6 +49,7 @@ "project_to_epsilon_machine", "project_to_mealy", "project_to_nmachine", + "learn_epsilon_machine_spectral", "spectral_singular_values", ] @@ -62,7 +66,7 @@ def _normalize_sequences(sequences: Iterable[Any]) -> list[tuple[Any, ...]]: Accepts either a single flat observation sequence (e.g. ``[0, 1, 0]``) or an iterable of sequences (e.g. ``[[0, 1], [1, 0]]``), mirroring - :func:`sofic.generators.hmm_inference.baum_welch`. + :func:`sofic.inference.hmm.baum_welch`. """ seqs = list(sequences) if not seqs: @@ -562,3 +566,59 @@ def register(state: MixedState) -> MixedState: initial_distribution=initial, observation_alphabet=frozenset(symbols), ) + + +def learn_epsilon_machine_spectral( + sequences: Iterable[Any] | None = None, + *, + word_probability: Callable[[Sequence[Any]], float] | None = None, + alphabet: Sequence[Any] | None = None, + rank: int | None = None, + prefix_length: int = 3, + suffix_length: int | None = None, + singular_value_threshold: float = 1e-3, + min_singular_value: float = 1e-12, + max_states: int = 10_000, +) -> EpsilonMachine: + """Reconstruct an ε-machine by spectral learning then mixed-state extraction. + + Learns a weighted finite automaton / observable-operator model from block + statistics :cite:`Balle2014,Hsu2012`, then extracts causal states as the + mixed states of those operators :cite:`Ellison2009`. When the learned + operators are non-negative this is a Mealy projection followed by + :meth:`~sofic.generators.epsilon_machine.EpsilonMachine.from_hmm`; signed + operators use mixed-state enumeration rather than a clustering heuristic. + + Parameters + ---------- + sequences + A single observed realization or an iterable of realizations. Ignored + when ``word_probability`` is given. + word_probability + Optional exact block-probability function ``f(word) -> float``. + ``alphabet`` is then required. + alphabet + Observation alphabet. Inferred from ``sequences`` when omitted. + rank + Number of latent states. When ``None`` the rank is chosen from the + Hankel singular-value spectrum. + prefix_length, suffix_length + Maximum lengths of the prefix and suffix bases. ``suffix_length`` + defaults to ``prefix_length``. + singular_value_threshold, min_singular_value + Cutoffs for automatic rank selection; see + :func:`~sofic.inference.spectral.learn_spectral_wfa`. + max_states + Safety cap on enumerated mixed states. + """ + model = learn_spectral_wfa( + sequences, + word_probability=word_probability, + alphabet=alphabet, + rank=rank, + prefix_length=prefix_length, + suffix_length=suffix_length, + singular_value_threshold=singular_value_threshold, + min_singular_value=min_singular_value, + ) + return project_to_epsilon_machine(model, max_states=max_states) diff --git a/sofic/properties.py b/sofic/properties.py index ed67507..bc211f8 100644 --- a/sofic/properties.py +++ b/sofic/properties.py @@ -214,7 +214,7 @@ def is_detailed_balance(model: StateMachine, *, rtol: float = 1e-8, atol: float """Return whether stationary labeled flows satisfy detailed balance.""" try: pi = np.asarray(model.stationary_distribution(), dtype=float) - except Exception: + except (AttributeError, ValueError, np.linalg.LinAlgError): pi, _transition = _initial_vector_and_transition(model) matrices = _labeled_or_internal_matrices(model) @@ -294,26 +294,10 @@ def transition_matrix( are ignored), otherwise all model states in iteration order. Returns the matrix together with the ordered state list defining its axes. """ - ordered = list(states) if states is not None else list(model.states()) - index = {state: i for i, state in enumerate(ordered)} - n = len(ordered) - from sofic.generators.prob import as_prob, has_symbolic, zeros - - edge_probs = [] - for state in ordered: - for transition in model.graph.out_transitions(state): - if transition.target in index: - edge_probs.append(transition.data.get(attr, 0.0)) - symbolic = has_symbolic(edge_probs) - matrix = zeros((n, n), symbolic=symbolic) - for state in ordered: - i = index[state] - for transition in model.graph.out_transitions(state): - j = index.get(transition.target) - if j is None: - continue - matrix[i, j] = as_prob(matrix[i, j]) + as_prob(transition.data.get(attr, 0.0)) - return matrix, ordered + from sofic.generators.matrices import accumulate_matrices + + matrices, ordered = accumulate_matrices(model, attr=attr, states=states, labels=(None,)) + return matrices[None], ordered def _initial_vector_and_transition(model: StateMachine) -> tuple[np.ndarray, np.ndarray]: @@ -330,10 +314,9 @@ def _labeled_or_internal_matrices(model: StateMachine) -> list[np.ndarray]: observation_alphabet = getattr(model, "observation_alphabet", None) to_mealy = getattr(model, "to_mealy", None) if observation_alphabet is not None and to_mealy is not None: - from sofic.generators.hmm_inference import _emission_transition_tensors_from_mealy + from sofic.generators.matrices import symbol_matrices - _pi, matrices = _emission_transition_tensors_from_mealy(to_mealy()) - return list(matrices.values()) + return list(symbol_matrices(to_mealy()).values()) _initial, transition = _initial_vector_and_transition(model) return [transition] diff --git a/sofic/serialization.py b/sofic/serialization.py index dac0024..1d6d4ea 100644 --- a/sofic/serialization.py +++ b/sofic/serialization.py @@ -50,11 +50,6 @@ def model_from_yaml(text: str, *, validate: bool = True) -> StateMachine: return model_from_dict(loaded, validate=validate) -def from_yaml(text: str, *, validate: bool = True) -> StateMachine: - """Alias for :func:`model_from_yaml`.""" - return model_from_yaml(text, validate=validate) - - def read_yaml(path: str | Path, *, validate: bool = True) -> StateMachine: """Read a sofic model from a YAML file.""" return model_from_yaml(Path(path).read_text(encoding="utf-8"), validate=validate) @@ -106,8 +101,6 @@ def _metadata_for(model: StateMachine, spec: _ModelSpec) -> dict[str, Any]: def _build_model(spec: _ModelSpec, graph: TransitionGraph, metadata: dict[str, Any]) -> StateMachine: - if spec.builder == "composite_vpa": - return spec.cls(operation=metadata["operation"], operands=metadata["operands"]) if spec.builder == "hidden_hmm": return spec.cls(graph=graph, observation_alphabet=_observation_alphabet(graph), **metadata) if spec.builder == "pfa": @@ -300,20 +293,19 @@ def _spec(cls: type[StateMachine], fields: tuple[str, ...], builder: str = "defa @cache def _specs() -> tuple[_ModelSpec, ...]: - from sofic.automata.atomaton import Atomaton, AtomicAutomaton, MaximizedPrimeAtomaton from sofic.automata.buchi import BuchiAutomaton + from sofic.automata.canonical.atomaton import Atomaton, AtomicAutomaton, MaximizedPrimeAtomaton + from sofic.automata.canonical.rfsa import CanonicalRFSA, ResidualFiniteStateAutomaton from sofic.automata.dfa import DFA from sofic.automata.nfa import NFA from sofic.automata.nwa import NestedWordAutomaton - from sofic.automata.rfsa import CanonicalRFSA, ResidualFiniteStateAutomaton from sofic.automata.subsequential import SubsequentialTransducer, WeightedFiniteStateTransducer from sofic.automata.transducers import MealyMachine, MooreMachine from sofic.automata.unifilar import UnifilarAutomaton from sofic.automata.vpa import ( - CallDrivenAutomaton, CanonicalVisiblyPushdownAutomaton, - CompositeVisiblyPushdownAutomaton, DeterministicVisiblyPushdownAutomaton, + ModularVisiblyPushdownAutomaton, MultipleEntryVisiblyPushdownAutomaton, SingleEntryVisiblyPushdownAutomaton, VisiblyPushdownAutomaton, @@ -339,10 +331,10 @@ def _specs() -> tuple[_ModelSpec, ...]: WheelerCover, ) from sofic.shifts.markov_dyck import MarkovDyckShift + from sofic.shifts.product_alphabet_shift import ProductAlphabetShift from sofic.shifts.sft import ShiftOfFiniteType from sofic.shifts.sofic import SoficShift from sofic.shifts.sofic_dyck import SoficDyckShift - from sofic.shifts.sofic_relation import SoficRelation from sofic.shifts.tmc import TopologicalMarkovChain labeled = ("input_alphabet", "initial_states", "accepting_states") @@ -405,11 +397,10 @@ def _specs() -> tuple[_ModelSpec, ...]: _spec(NestedWordAutomaton, nwa), _spec(VisiblyPushdownAutomaton, vpa), _spec(DeterministicVisiblyPushdownAutomaton, vpa), - _spec(CallDrivenAutomaton, cda), + _spec(ModularVisiblyPushdownAutomaton, cda), _spec(MultipleEntryVisiblyPushdownAutomaton, (*cda, "entry_states")), _spec(SingleEntryVisiblyPushdownAutomaton, (*cda, "entry_states")), _spec(CanonicalVisiblyPushdownAutomaton, (*vpa, "summary_representatives")), - _spec(CompositeVisiblyPushdownAutomaton, ("operation", "operands"), builder="composite_vpa"), _spec(StochasticModel, stochastic), _spec(HiddenMarkovModel, hidden, builder="hidden_hmm"), _spec(MarkovChain, stochastic), @@ -444,7 +435,7 @@ def _specs() -> tuple[_ModelSpec, ...]: _spec(QuasiRealization, (*quasi, "pi", "tau", "symbol_maps")), _spec(SymbolicModel, symbolic), _spec(SoficShift, symbolic), - _spec(SoficRelation, symbolic), + _spec(ProductAlphabetShift, symbolic), _spec(TopologicalMarkovChain, symbolic), _spec(ShiftOfFiniteType, (*symbolic, "_forbidden_words", "_has_forbidden_word_spec"), builder="sft"), _spec(SoficDyckShift, dyck), diff --git a/sofic/shifts/__init__.py b/sofic/shifts/__init__.py index 157ad34..1012474 100644 --- a/sofic/shifts/__init__.py +++ b/sofic/shifts/__init__.py @@ -17,11 +17,11 @@ shift_to_dyck_graph_string, ) from sofic.shifts.markov_dyck import MarkovDyckShift +from sofic.shifts.product_alphabet_shift import ProductAlphabetShift from sofic.shifts.sft import ShiftOfFiniteType from sofic.shifts.sliding_block_code import SlidingBlockCode, full_shift from sofic.shifts.sofic import SoficShift from sofic.shifts.sofic_dyck import SoficDyckShift -from sofic.shifts.sofic_relation import SoficRelation from sofic.shifts.textile import TextileSystem from sofic.shifts.tmc import TopologicalMarkovChain from sofic.shifts.wheeler import ( @@ -49,7 +49,7 @@ "SoficShift", "shift_to_dyck_graph_string", "SoficDyckShift", - "SoficRelation", + "ProductAlphabetShift", "SymbolicModel", "TextileSystem", "TopologicalMarkovChain", diff --git a/sofic/shifts/algorithms.py b/sofic/shifts/algorithms.py index 2214cf6..514855a 100644 --- a/sofic/shifts/algorithms.py +++ b/sofic/shifts/algorithms.py @@ -8,7 +8,7 @@ import numpy as np -from sofic.graph import ATTR_SYMBOL +from sofic.graph import ATTR_MULTIPLICITY, ATTR_SYMBOL from sofic.shifts.base import SymbolicModel @@ -54,16 +54,19 @@ def adjacency_matrix(model: SymbolicModel) -> tuple[np.ndarray, tuple[Hashable, for transition in model.transitions(): i = index[transition.source] j = index[transition.target] - matrix[i, j] += 1.0 + matrix[i, j] += float(transition.data.get(ATTR_MULTIPLICITY, 1)) return matrix, states def topological_entropy_from_matrix(matrix: np.ndarray) -> float: + """Return ``log2`` of the spectral radius of ``matrix`` (bits per symbol).""" if matrix.size == 0: return 0.0 eigenvalues = np.linalg.eigvals(matrix) spectral_radius = float(np.max(np.abs(eigenvalues))) - return float(np.log(max(spectral_radius, 0.0))) + if spectral_radius <= 0.0: + return 0.0 + return float(np.log2(spectral_radius)) def _forward_reachable(model: SymbolicModel) -> set[Hashable]: diff --git a/sofic/shifts/cover_construction.py b/sofic/shifts/cover_construction.py index cb4daf1..ddbf5f1 100644 --- a/sofic/shifts/cover_construction.py +++ b/sofic/shifts/cover_construction.py @@ -1,76 +1,204 @@ -"""Fischer and Krieger cover constructions.""" +"""Fischer and Krieger cover constructions. + +Both covers are computed exactly from a presentation ``G`` with vertex set +``Q``. Write ``S . w`` for the set of vertices reached from ``S`` along paths +labeled ``w``, and ``F(S)`` for the follower set (future language) of ``S``. + +* The **right Fischer cover** of an irreducible sofic shift is its unique + minimal right-resolving presentation :cite:`Fischer1975` + :cite:`LindMarcus1995`. It is the unique terminal strongly connected + component of the subset construction from ``Q`` after merging subsets with + equal follower sets: an intrinsically synchronizing word ``m`` sends every + subset to the follower class ``F(m)``, so that class is reachable from all + others. +* The **right Krieger cover** (future cover) has one vertex per follower set + ``F(x^-)`` of a left-infinite ray, with ``F(x^-) --a--> F(x^- a)`` + :cite:`Krieger1984` :cite:`LindMarcus1995`. ``F(x^-) = F(T(x^-))`` where + ``T(x^-) = Q . s`` for every long enough suffix ``s`` of ``x^-``. Reading a + ray right to left composes path relations ``rho_{cs} = rho_c o rho_s`` in a + finite monoid, so the sets ``T(x^-)`` are exactly the images ``Q . rho`` of + relations ``rho`` that lie on a cycle reachable from the identity. + +The left covers are the mirror images: the right cover of the reversed shift, +reversed back. +""" from __future__ import annotations -from collections import defaultdict +from collections import defaultdict, deque +from collections.abc import Hashable, Iterable from typing import Any +import networkx as nx + +from sofic.exceptions import SoficValidationError from sofic.graph import ATTR_SYMBOL, TransitionGraph from sofic.shifts.covers import LeftFischerCover, LeftKriegerCover, RightFischerCover, RightKriegerCover from sofic.shifts.sofic import SoficShift from sofic.states import sequential_labels +Subset = frozenset[Hashable] +Relation = frozenset[tuple[Hashable, Hashable]] -def _follower_language(shift: SoficShift, vertex: Any, max_len: int = 8) -> frozenset[tuple[Any, ...]]: - from collections import deque - seen: set[tuple[Any, ...]] = set() - queue: deque[tuple[Any, tuple[Any, ...]]] = deque([(vertex, ())]) +def _labeled_successors(shift: SoficShift) -> dict[Hashable, dict[Any, set[Hashable]]]: + successors: dict[Hashable, dict[Any, set[Hashable]]] = {state: defaultdict(set) for state in shift.states()} + for transition in shift.transitions(): + symbol = transition.data.get(ATTR_SYMBOL) + if symbol is not None: + successors[transition.source][symbol].add(transition.target) + return successors + + +def _subset_automaton( + successors: dict[Hashable, dict[Any, set[Hashable]]], +) -> dict[Subset, dict[Any, Subset]]: + """Deterministic subset automaton reachable from the full vertex set.""" + start: Subset = frozenset(successors) + delta: dict[Subset, dict[Any, Subset]] = {} + queue = deque([start]) while queue: - state, prefix = queue.popleft() - if len(prefix) > max_len: + subset = queue.popleft() + if subset in delta: continue - if prefix: - seen.add(prefix) - for transition in shift.graph.out_transitions(state): - symbol = transition.data.get(ATTR_SYMBOL) - if symbol is None: - continue - queue.append((transition.target, prefix + (symbol,))) - return frozenset(seen) - - -def _build_left_fischer(shift: SoficShift) -> LeftFischerCover: - followers: dict[Any, frozenset[tuple[Any, ...]]] = { - vertex: _follower_language(shift, vertex) for vertex in shift.states() - } - classes: dict[frozenset[tuple[Any, ...]], list[Any]] = defaultdict(list) - for vertex, language in followers.items(): - classes[language].append(vertex) - + moves: dict[Any, set[Hashable]] = defaultdict(set) + for state in subset: + for symbol, targets in successors[state].items(): + moves[symbol] |= targets + delta[subset] = {symbol: frozenset(targets) for symbol, targets in moves.items() if targets} + queue.extend(target for target in delta[subset].values() if target not in delta) + return delta + + +def _follower_classes(delta: dict[Subset, dict[Any, Subset]]) -> dict[Subset, int]: + """Moore refinement: subsets with equal follower sets share a class.""" + block = dict.fromkeys(delta, 0) + while True: + signatures = { + subset: (block[subset], tuple(sorted(((repr(a), block[t]) for a, t in moves.items())))) + for subset, moves in delta.items() + } + ids: dict[Any, int] = {} + refined = {subset: ids.setdefault(signature, len(ids)) for subset, signature in signatures.items()} + if len(ids) == len(set(block.values())): + return refined + block = refined + + +def _labels(count: int) -> tuple[Hashable, ...]: + return sequential_labels(count) if count <= 26 else tuple(range(count)) + + +def _quotient_shift( + cls: type[SoficShift], + shift: SoficShift, + vertices: Iterable[Subset], + delta: dict[Subset, dict[Any, Subset]], + classes: dict[Subset, int], +) -> SoficShift: + keep = set(vertices) + used = sorted({classes[subset] for subset in keep}) + name = dict(zip(used, _labels(len(used)), strict=True)) graph = TransitionGraph() - class_for_vertex = {vertex: language for vertex, language in followers.items()} - state_ids = {language: sequential_labels(len(classes))[index] for index, language in enumerate(classes)} - for _language, state_id in state_ids.items(): - graph.add_state(state_id) - - for transition in shift.transitions(): - source_lang = class_for_vertex[transition.source] - target_lang = class_for_vertex[transition.target] - symbol = transition.data.get(ATTR_SYMBOL) - graph.add_transition(state_ids[source_lang], state_ids[target_lang], **{ATTR_SYMBOL: symbol}) - - return LeftFischerCover( - graph=graph, - symbol_alphabet=shift.symbol_alphabet, - ) - - -def left_fischer_from_sofic(shift: SoficShift) -> LeftFischerCover: - return _build_left_fischer(shift.trim_transient()) - - -def right_fischer_from_sofic(shift: SoficShift) -> RightFischerCover: - left = left_fischer_from_sofic(shift.reverse()) - return RightFischerCover( - graph=left.graph.copy(), - symbol_alphabet=left.symbol_alphabet, - ) - - -def left_krieger_from_sofic(shift: SoficShift) -> LeftKriegerCover: - raise NotImplementedError("Left Krieger cover construction is not yet implemented; use left_fischer_from_sofic") + for class_id in used: + graph.add_state(name[class_id]) + edges = { + (classes[subset], symbol, classes[target]) + for subset in keep + for symbol, target in delta[subset].items() + if target in keep + } + for source, symbol, target in sorted(edges, key=repr): + graph.add_transition(name[source], name[target], **{ATTR_SYMBOL: symbol}) + return cls(graph=graph, symbol_alphabet=shift.symbol_alphabet) + + +def _mirror(cls: type[SoficShift], cover: SoficShift) -> SoficShift: + return cls(graph=cover.graph.reverse(), symbol_alphabet=cover.symbol_alphabet) + + +def right_fischer_cover(shift: SoficShift) -> RightFischerCover: + """Return the minimal right-resolving presentation of an irreducible sofic shift. + + Raises :class:`~sofic.exceptions.SoficValidationError` when the shift is + reducible (more than one terminal component), in which case a minimal + right-resolving presentation need not be unique :cite:`LindMarcus1995`. + """ + trimmed = shift.trim_transient() + delta = _subset_automaton(_labeled_successors(trimmed)) + if not delta or not any(delta.values()): + return RightFischerCover(symbol_alphabet=shift.symbol_alphabet) + classes = _follower_classes(delta) + + quotient = nx.DiGraph() + quotient.add_nodes_from(set(classes.values())) + for subset, moves in delta.items(): + for target in moves.values(): + quotient.add_edge(classes[subset], classes[target]) + condensation = nx.condensation(quotient) + terminal = [node for node in condensation.nodes if condensation.out_degree(node) == 0] + if len(terminal) != 1: + raise SoficValidationError("the Fischer cover is defined for irreducible sofic shifts; this shift is reducible") + members = set(condensation.nodes[terminal[0]]["members"]) + if not any(classes[s] in members and delta[s] for s in delta): + return RightFischerCover(symbol_alphabet=shift.symbol_alphabet) + vertices = [subset for subset in delta if classes[subset] in members] + return _quotient_shift(RightFischerCover, shift, vertices, delta, classes) + + +def left_fischer_cover(shift: SoficShift) -> LeftFischerCover: + """Return the minimal left-resolving presentation (mirror of the right Fischer cover).""" + return _mirror(LeftFischerCover, right_fischer_cover(shift.reverse())) + + +def _ray_terminal_sets(successors: dict[Hashable, dict[Any, set[Hashable]]]) -> set[Subset]: + """Return ``{T(x^-)}``: images of path relations lying on reachable cycles.""" + symbols = {symbol for moves in successors.values() for symbol in moves} + letter: dict[Any, Relation] = { + symbol: frozenset((p, q) for p, moves in successors.items() for q in moves.get(symbol, ())) + for symbol in symbols + } + identity: Relation = frozenset((q, q) for q in successors) + def prepend(symbol: Any, relation: Relation) -> Relation: + after: dict[Hashable, set[Hashable]] = defaultdict(set) + for q, r in relation: + after[q].add(r) + return frozenset((p, r) for p, q in letter[symbol] for r in after.get(q, ())) -def right_krieger_from_sofic(shift: SoficShift) -> RightKriegerCover: - raise NotImplementedError("Right Krieger cover construction is not yet implemented; use right_fischer_from_sofic") + graph = nx.DiGraph() + graph.add_node(identity) + queue = deque([identity]) + while queue: + relation = queue.popleft() + for symbol in symbols: + extended = prepend(symbol, relation) + if not extended: + continue + if extended not in graph: + queue.append(extended) + graph.add_edge(relation, extended) + + recurrent: set[Relation] = set() + for component in nx.strongly_connected_components(graph): + node = next(iter(component)) + if len(component) > 1 or graph.has_edge(node, node): + recurrent |= component + return {frozenset(r for _q, r in relation) for relation in recurrent} + + +def right_krieger_cover(shift: SoficShift) -> RightKriegerCover: + """Return the right Krieger (future) cover of ``shift``.""" + trimmed = shift.trim_transient() + successors = _labeled_successors(trimmed) + if not successors: + return RightKriegerCover(symbol_alphabet=shift.symbol_alphabet) + delta = _subset_automaton(successors) + classes = _follower_classes(delta) + vertices = [subset for subset in _ray_terminal_sets(successors) if subset in delta] + return _quotient_shift(RightKriegerCover, shift, vertices, delta, classes) + + +def left_krieger_cover(shift: SoficShift) -> LeftKriegerCover: + """Return the left Krieger (past) cover: the mirror of the right Krieger cover.""" + return _mirror(LeftKriegerCover, right_krieger_cover(shift.reverse())) diff --git a/sofic/shifts/covers.py b/sofic/shifts/covers.py index 839e883..f9099d0 100644 --- a/sofic/shifts/covers.py +++ b/sofic/shifts/covers.py @@ -11,40 +11,40 @@ class LeftFischerCover(SoficShift): """Left Fischer cover presentation.""" @classmethod - def from_sofic(cls, shift: SoficShift, **kwargs: Any) -> LeftFischerCover: - from sofic.shifts.cover_construction import left_fischer_from_sofic + def from_presentation(cls, shift: SoficShift, **kwargs: Any) -> LeftFischerCover: + from sofic.shifts.cover_construction import left_fischer_cover - return left_fischer_from_sofic(shift) + return left_fischer_cover(shift) class RightFischerCover(SoficShift): """Right Fischer cover presentation.""" @classmethod - def from_sofic(cls, shift: SoficShift, **kwargs: Any) -> RightFischerCover: - from sofic.shifts.cover_construction import right_fischer_from_sofic + def from_presentation(cls, shift: SoficShift, **kwargs: Any) -> RightFischerCover: + from sofic.shifts.cover_construction import right_fischer_cover - return right_fischer_from_sofic(shift) + return right_fischer_cover(shift) class LeftKriegerCover(SoficShift): """Left Krieger cover presentation.""" @classmethod - def from_sofic(cls, shift: SoficShift, **kwargs: Any) -> LeftKriegerCover: - from sofic.shifts.cover_construction import left_krieger_from_sofic + def from_presentation(cls, shift: SoficShift, **kwargs: Any) -> LeftKriegerCover: + from sofic.shifts.cover_construction import left_krieger_cover - return left_krieger_from_sofic(shift) + return left_krieger_cover(shift) class RightKriegerCover(SoficShift): """Right Krieger cover presentation.""" @classmethod - def from_sofic(cls, shift: SoficShift, **kwargs: Any) -> RightKriegerCover: - from sofic.shifts.cover_construction import right_krieger_from_sofic + def from_presentation(cls, shift: SoficShift, **kwargs: Any) -> RightKriegerCover: + from sofic.shifts.cover_construction import right_krieger_cover - return right_krieger_from_sofic(shift) + return right_krieger_cover(shift) class WheelerCover(SoficShift): @@ -57,7 +57,7 @@ class WheelerCover(SoficShift): """ @classmethod - def from_sofic(cls, shift: SoficShift, **kwargs: Any) -> WheelerCover: + def from_presentation(cls, shift: SoficShift, **kwargs: Any) -> WheelerCover: from sofic.shifts.wheeler import wheeler_cover return wheeler_cover(shift, **kwargs) diff --git a/sofic/shifts/sofic_relation.py b/sofic/shifts/product_alphabet_shift.py similarity index 90% rename from sofic/shifts/sofic_relation.py rename to sofic/shifts/product_alphabet_shift.py index 3bc1d8a..add06b2 100644 --- a/sofic/shifts/sofic_relation.py +++ b/sofic/shifts/product_alphabet_shift.py @@ -1,7 +1,7 @@ -"""Sofic relations: subshifts over a product alphabet ``X x Y``. +"""Product-alphabet shifts: sofic subshifts over a product alphabet ``X x Y``. The topological support of a transducer is a subshift of the product shift on -``X x Y`` -- a "sofic relation" whose input and output projections are the +``X x Y`` -- a sofic relation whose input and output projections are the transducer's domain and range subshifts. This is the symbolic-dynamics reading of a transducer, complementary to the sliding block code (Lind & Marcus, *An Introduction to Symbolic Dynamics and Coding* (1995), ch. 6). @@ -15,11 +15,11 @@ from sofic.shifts.sofic import SoficShift -class SoficRelation(SoficShift): +class ProductAlphabetShift(SoficShift): """A sofic subshift whose symbols are ``(input, output)`` pairs.""" @classmethod - def from_transducer(cls, transducer: Any) -> SoficRelation: + def from_transducer(cls, transducer: Any) -> ProductAlphabetShift: """Build the topological support of a transducer (probabilities dropped).""" relation = cls() pairs: set[tuple[Any, Any]] = set() diff --git a/sofic/shifts/sft_construction.py b/sofic/shifts/sft_construction.py index 05fe2a9..559d433 100644 --- a/sofic/shifts/sft_construction.py +++ b/sofic/shifts/sft_construction.py @@ -4,6 +4,7 @@ from typing import Any +from sofic.exceptions import SoficValidationError from sofic.graph import ATTR_SYMBOL, TransitionGraph from sofic.shifts.sft import ShiftOfFiniteType @@ -14,7 +15,12 @@ def from_forbidden_words( *, max_states: int = 256, ) -> ShiftOfFiniteType: - """Build an SFT presentation via a follower automaton on allowed prefixes.""" + """Build an SFT presentation via a follower automaton on allowed prefixes. + + Raises :class:`~sofic.exceptions.SoficValidationError` when the presentation + would need more than ``max_states`` states rather than returning a truncated + automaton. + """ alphabet = tuple(symbol_alphabet) forbidden_set = set(forbidden) max_len = max((len(word) for word in forbidden_set), default=0) @@ -30,16 +36,16 @@ def is_allowed(prefix: tuple[Any, ...]) -> bool: while queue: prefix = queue.pop(0) - if len(seen) >= max_states: - break for symbol in alphabet: extended = prefix + (symbol,) if not is_allowed(extended): continue - trimmed = extended - if max_len > 0: - trimmed = extended[-max_len:] + trimmed = extended[-max_len:] if max_len > 0 else () if trimmed not in seen: + if len(seen) >= max_states: + raise SoficValidationError( + f"SFT presentation needs more than max_states={max_states} states; raise max_states" + ) seen.add(trimmed) graph.add_state(trimmed) queue.append(trimmed) diff --git a/sofic/shifts/sliding_block_code.py b/sofic/shifts/sliding_block_code.py index 3f249b1..577e91c 100644 --- a/sofic/shifts/sliding_block_code.py +++ b/sofic/shifts/sliding_block_code.py @@ -73,22 +73,43 @@ def apply_word(self, word: Sequence[Any]) -> tuple[Any, ...]: def apply(self, shift: Any) -> SoficShift: """Return the image subshift ``Phi(shift)`` as a sofic presentation. - Uses the higher-block construction: vertices are allowed - ``(window - 1)``-blocks of ``shift`` and each allowed ``window``-block - contributes an edge labeled by its image symbol. + Vertices pair a vertex ``q`` of ``shift``'s presentation with the last + ``window - 1`` symbols read along a path into ``q``; each edge of + ``shift`` out of ``q`` reading ``a`` emits ``Phi(context + a)``. Tracking + the presentation vertex keeps every constraint of ``shift``, not just + those visible in ``window``-blocks. Raises :class:`ValueError` when a + ``window``-block of ``shift`` is missing from ``block_map``. """ + outgoing: dict[Any, list[tuple[Any, Any]]] = {state: [] for state in shift.states()} + for transition in shift.transitions(): + symbol = transition.data.get(ATTR_SYMBOL) + if symbol is not None: + outgoing[transition.source].append((symbol, transition.target)) + + context_length = self.window - 1 + frontier = {(state, ()) for state in outgoing} + for _ in range(context_length): + frontier = { + (target, (*context, symbol)) for state, context in frontier for symbol, target in outgoing[state] + } + image = SoficShift(symbol_alphabet=frozenset(self.output_alphabet)) - blocks = list(shift.factor_language(self.window)) - vertices = {block[:-1] for block in blocks} | {block[1:] for block in blocks} - for vertex in vertices: - image.graph.add_state(vertex) + seen = set(frontier) + queue = list(frontier) used_outputs: set[Any] = set() - for block in blocks: - output = self.block_map.get(block) - if output is None: - continue - image.add_transition(block[:-1], block[1:], output) - used_outputs.add(output) + while queue: + state, context = queue.pop() + image.graph.add_state((state, context)) + for symbol, target in outgoing[state]: + block = (*context, symbol) + if block not in self.block_map: + raise ValueError(f"block {block!r} of the shift is not in block_map") + successor = (target, block[1:]) + image.add_transition((state, context), successor, self.block_map[block]) + used_outputs.add(self.block_map[block]) + if successor not in seen: + seen.add(successor) + queue.append(successor) image.symbol_alphabet = frozenset(used_outputs) return image.trim_transient() diff --git a/sofic/shifts/textile.py b/sofic/shifts/textile.py index 0a1f821..4c61f97 100644 --- a/sofic/shifts/textile.py +++ b/sofic/shifts/textile.py @@ -35,19 +35,19 @@ def to_transducer(self) -> Any: """Return the underlying Mealy machine.""" return self._transducer.copy() - def to_sofic_relation(self) -> Any: + def to_product_alphabet_shift(self) -> Any: """Return the product-alphabet subshift of paired labels.""" - from sofic.shifts.sofic_relation import SoficRelation + from sofic.shifts.product_alphabet_shift import ProductAlphabetShift - return SoficRelation.from_transducer(self._transducer) + return ProductAlphabetShift.from_transducer(self._transducer) def input_shift(self) -> SoficShift: """Return the input subshift (``p`` labeling).""" - return self.to_sofic_relation().input_shift() + return self.to_product_alphabet_shift().input_shift() def output_shift(self) -> SoficShift: """Return the output subshift (``q`` labeling).""" - return self.to_sofic_relation().output_shift() + return self.to_product_alphabet_shift().output_shift() def induced_code(self, *, max_window: int = 4) -> SlidingBlockCode: """Return the induced sliding block code (memory only), if it has finite window. diff --git a/sofic/shifts/topological_anatomy.py b/sofic/shifts/topological_anatomy.py index 4615b62..a2f07a9 100644 --- a/sofic/shifts/topological_anatomy.py +++ b/sofic/shifts/topological_anatomy.py @@ -85,18 +85,17 @@ def _right_resolving(shift: SoficShift) -> SoficShift: """Return a right-resolving (unifilar) presentation of ``shift``. If ``shift`` is already unifilar it is returned unchanged. Otherwise the right - Fischer cover (:meth:`~sofic.shifts.covers.RightFischerCover.from_sofic`) is + Fischer cover (:meth:`~sofic.shifts.covers.RightFischerCover.from_presentation`) is built and its duplicate labeled edges merged (:func:`_dedup_symbol_edges`). - The cover construction uses a bounded follower language, so it is not - guaranteed to determinize every presentation; if the result is still not - unifilar a :class:`~sofic.exceptions.UnifilarityError` is raised asking for a - right-resolving input. + The cover is exact for irreducible shifts; a reducible non-unifilar + presentation raises :class:`~sofic.exceptions.SoficValidationError` from the + cover construction. """ if shift.is_unifilar(): return shift from sofic.shifts.covers import RightFischerCover - cover = _dedup_symbol_edges(RightFischerCover.from_sofic(shift)) + cover = _dedup_symbol_edges(RightFischerCover.from_presentation(shift)) if not cover.is_unifilar(): raise UnifilarityError( "could not derive a right-resolving presentation of the sofic shift " diff --git a/sofic/testing/strategies.py b/sofic/testing/strategies.py index cab8e14..d84efb3 100644 --- a/sofic/testing/strategies.py +++ b/sofic/testing/strategies.py @@ -11,7 +11,7 @@ from typing import Any from sofic.automata.dfa import DFA -from sofic.automata.icdfa import icdfa_string_to_dfa, iter_icdfa_empty_strings +from sofic.automata.enumeration.icdfa import icdfa_string_to_dfa, iter_icdfa_empty_strings from sofic.generators.epsilon_machine import EpsilonMachine from sofic.generators.topological_epsilon_enumeration import ( idfa_string_to_epsilon_machine, diff --git a/sofic/viz/_format.py b/sofic/viz/_format.py index 7b8ed91..32c9fc4 100644 --- a/sofic/viz/_format.py +++ b/sofic/viz/_format.py @@ -3,10 +3,10 @@ from __future__ import annotations from collections.abc import Mapping, Sequence +from fractions import Fraction from typing import Any from sofic.viz import _labels -from sofic.viz._rational import two_digit_rational def dot_escape(text: str) -> str: @@ -34,42 +34,26 @@ def format_prob(value: float, *, precision: int = 3) -> str: return dot_escape(f"{value:.{precision}g}") +def _dot_fraction(frac: Fraction) -> str: + return dot_escape(f"{frac.numerator}/{frac.denominator}") + + +def _dot_sympy(expr: Any) -> str: + import sympy as sp + + return dot_escape(sp.sstr(expr)) + + def format_prob_rational(value: float, *, precision: int = 3) -> str: """Format a probability as p/q when exact with two-digit numerator and denominator.""" - if value <= 0.0: - return "0" - if value >= 1.0: - return "1" - frac = two_digit_rational(value) - if frac is not None: - if frac.numerator == frac.denominator: - return "1" - return dot_escape(f"{frac.numerator}/{frac.denominator}") - return format_prob(value, precision=precision) + return format_prob_label(float(value), precision=precision) def format_prob_label(value: Any, *, precision: int = 3) -> str: """Format a probability for Graphviz edge/π labels (float or sympy Expr).""" - try: - from sofic.generators.prob import is_symbolic, simplify_prob - except ImportError: # pragma: no cover - - def is_symbolic(_v: Any) -> bool: - return False - - def simplify_prob(v: Any) -> Any: - return v - - if is_symbolic(value): - simplified = simplify_prob(value) - try: - import sympy as sp - - text = sp.sstr(simplified) - except Exception: - text = str(simplified) - return dot_escape(text) - return format_prob_rational(float(value), precision=precision) + return _labels.format_prob( + value, escape=dot_escape, fraction=_dot_fraction, symbolic=_dot_sympy, precision=precision + ) def format_distribution(dist: Mapping[Any, Any], *, precision: int = 3) -> str: diff --git a/sofic/viz/_labels.py b/sofic/viz/_labels.py index 0cf3b3a..8f3c91f 100644 --- a/sofic/viz/_labels.py +++ b/sofic/viz/_labels.py @@ -1,18 +1,37 @@ -"""Backend-agnostic skeletons for state/symbol label formatting. +"""Backend-agnostic skeletons for state/symbol/probability label formatting. The graphviz and LaTeX backends share the same structural recursion for formatting states (tuples recurse, frozensets render as sorted ``{...}``, -``EPSILON`` becomes a glyph) and symbols; they differ only in the leaf escape -function and the epsilon glyph. These helpers capture that shared shape. +``EPSILON`` becomes a glyph), symbols, and probabilities (symbolic +expressions, clamped ``0`` / ``1``, exact two-digit fractions, else +``precision`` significant digits); they differ only in the leaf escape +function, the epsilon glyph, and how fractions / sympy expressions are +spelled. These helpers capture that shared shape. """ from __future__ import annotations from collections.abc import Callable +from fractions import Fraction from typing import Any from sofic.graph import EPSILON +_TWO_DIGIT_RATIONAL_ATOL = 1e-9 +_MAX_TWO_DIGIT_RATIONAL = 99 + + +def two_digit_rational(value: float, *, atol: float = _TWO_DIGIT_RATIONAL_ATOL) -> Fraction | None: + """Return a reduced rational with 1 <= p, q <= 99 when ``value`` matches exactly.""" + if value <= 0.0 or value >= 1.0: + return None + frac = Fraction(value).limit_denominator(_MAX_TWO_DIGIT_RATIONAL) + if abs(float(frac) - value) >= atol: + return None + if not (1 <= frac.numerator <= _MAX_TWO_DIGIT_RATIONAL and 1 <= frac.denominator <= _MAX_TWO_DIGIT_RATIONAL): + return None + return frac + def format_state(state: Any, *, escape: Callable[[str], str], epsilon: str) -> str: """Format a (possibly nested) state using ``escape`` for leaves.""" @@ -32,3 +51,39 @@ def format_symbol(symbol: Any, *, escape: Callable[[str], str], epsilon: str) -> if symbol is EPSILON: return epsilon return escape(str(symbol)) + + +def format_prob( + value: Any, + *, + escape: Callable[[str], str], + fraction: Callable[[Fraction], str], + symbolic: Callable[[Any], str], + precision: int = 3, +) -> str: + """Format a probability (float or sympy Expr). + + ``symbolic`` spells a simplified sympy expression and ``fraction`` an exact + two-digit rational, each returning backend-ready text. Decimals and the + ``str`` fallback (used when ``symbolic`` raises) pass through ``escape``. + """ + from sofic.generators.prob import is_symbolic, simplify_prob + + if is_symbolic(value): + simplified = simplify_prob(value) + try: + return symbolic(simplified) + except Exception: + return escape(str(simplified)) + + numeric = float(value) + if numeric <= 0.0: + return "0" + if numeric >= 1.0: + return "1" + frac = two_digit_rational(numeric) + if frac is not None: + if frac.numerator == frac.denominator: + return "1" + return fraction(frac) + return escape(f"{numeric:.{precision}g}") diff --git a/sofic/viz/_rational.py b/sofic/viz/_rational.py deleted file mode 100644 index e961660..0000000 --- a/sofic/viz/_rational.py +++ /dev/null @@ -1,20 +0,0 @@ -"""Shared rational-number helper for viz label formatting.""" - -from __future__ import annotations - -from fractions import Fraction - -_TWO_DIGIT_RATIONAL_ATOL = 1e-9 -_MAX_TWO_DIGIT_RATIONAL = 99 - - -def two_digit_rational(value: float, *, atol: float = _TWO_DIGIT_RATIONAL_ATOL) -> Fraction | None: - """Return a reduced rational with 1 <= p, q <= 99 when ``value`` matches exactly.""" - if value <= 0.0 or value >= 1.0: - return None - frac = Fraction(value).limit_denominator(_MAX_TWO_DIGIT_RATIONAL) - if abs(float(frac) - value) >= atol: - return None - if not (1 <= frac.numerator <= _MAX_TWO_DIGIT_RATIONAL and 1 <= frac.denominator <= _MAX_TWO_DIGIT_RATIONAL): - return None - return frac diff --git a/sofic/viz/_tikz_format.py b/sofic/viz/_tikz_format.py index 5a5a8f1..74b591f 100644 --- a/sofic/viz/_tikz_format.py +++ b/sofic/viz/_tikz_format.py @@ -3,10 +3,12 @@ from __future__ import annotations from collections.abc import Sequence +from fractions import Fraction from typing import Any from sofic.viz import _labels -from sofic.viz._rational import _TWO_DIGIT_RATIONAL_ATOL, two_digit_rational + +_HALF = Fraction(1, 2) _LATEX_SPECIAL = { "\\": r"\textbackslash{}", @@ -59,64 +61,25 @@ def format_symbol_latex(symbol: Any) -> str: return _labels.format_symbol(symbol, escape=_latex_arg, epsilon=r"\varepsilon") -def format_prob_latex(value: Any, *, precision: int = 3) -> str: - """Format a probability for Vaucanson edge labels (float or sympy Expr).""" - try: - from sofic.generators.prob import is_symbolic, simplify_prob - except ImportError: # pragma: no cover - is_symbolic = lambda _v: False # noqa: E731 - simplify_prob = lambda v: v # noqa: E731 - - if is_symbolic(value): - simplified = simplify_prob(value) - try: - import sympy as sp - - return latex_escape(sp.latex(simplified)) - except Exception: - return latex_escape(str(simplified)) - - numeric = float(value) - if numeric <= 0.0: - return "0" - if numeric >= 1.0: - return "1" - if abs(numeric - 0.5) < _TWO_DIGIT_RATIONAL_ATOL: +def _latex_fraction(frac: Fraction) -> str: + if frac == _HALF: return r"\half" - frac = two_digit_rational(numeric) - if frac is not None: - if frac.numerator == frac.denominator: - return "1" - return rf"\nicefrac{{{frac.numerator}}}{{{frac.denominator}}}" - return latex_escape(f"{numeric:.{precision}g}") - + return rf"\nicefrac{{{frac.numerator}}}{{{frac.denominator}}}" -def format_symbol_macro(symbol: Any) -> str: - return rf"\Symbol{{{format_symbol_latex(symbol)}}}" +def _latex_sympy(expr: Any) -> str: + import sympy as sp -def format_edge_latex(symbol: Any, prob: Any) -> str: - """Return ``$\\Edge{sym}{prob}$`` math content.""" - return rf"$\Edge{{{format_symbol_latex(symbol)}}}{{{format_prob_latex(prob)}}}$" + return sp.latex(expr) -def format_tedge_latex(forward: Any, reverse: Any, prob: Any) -> str: - """Return ``$\\TEdge{f}{r}{prob}$`` math content.""" - return ( - rf"$\TEdge{{{format_symbol_latex(forward)}}}" - rf"{{{format_symbol_latex(reverse)}}}" - rf"{{{format_prob_latex(prob)}}}$" +def format_prob_latex(value: Any, *, precision: int = 3) -> str: + """Format a probability for Vaucanson edge labels (float or sympy Expr).""" + return _labels.format_prob( + value, escape=latex_escape, fraction=_latex_fraction, symbolic=_latex_sympy, precision=precision ) -def format_symbol_only_latex(symbol: Any) -> str: - return rf"$\Symbol{{{format_symbol_latex(symbol)}}}$" - - -def format_transducer_edge_latex(input_symbol: Any, output_symbol: Any | None = None) -> str: - if output_symbol is None: - return format_symbol_only_latex(input_symbol) - return ( - rf"$\Symbol{{{format_symbol_latex(input_symbol)}}}" - rf"\mid\Symbol{{{format_symbol_latex(output_symbol)}}}$" - ) +def format_edge_latex(symbol: Any, prob: Any) -> str: + """Return ``$\\Edge{sym}{prob}$`` math content.""" + return rf"$\Edge{{{format_symbol_latex(symbol)}}}{{{format_prob_latex(prob)}}}$" diff --git a/sofic/viz/tikz.py b/sofic/viz/tikz.py index cc41b84..353499d 100644 --- a/sofic/viz/tikz.py +++ b/sofic/viz/tikz.py @@ -18,18 +18,17 @@ PART_KIND, PART_MATCH_TAG, PART_MULTIPLICITY, - PART_OUTPUT, PART_PROB, PART_QUASIPROB, PART_STACK, PART_SYMBOL, STYLE_DYCK, STYLE_EDGE, - STYLE_PROB_ONLY, - STYLE_SYMBOL_ONLY, + STYLE_FALLBACK, STYLE_TMC, STYLE_TRANSDUCER, STYLE_VPA, + EdgePart, edge_spec, part_value, ) @@ -41,8 +40,6 @@ format_state_latex, format_state_tikz_node, format_symbol_latex, - format_symbol_only_latex, - format_transducer_edge_latex, latex_escape, ) from sofic.viz._tikz_layout import ( @@ -72,77 +69,39 @@ def _format_dyck_match_tag_latex(tag: str) -> str: return latex_escape(tag) +# Symbols render bare in these styles; elsewhere they are wrapped in ``\Symbol{...}``. +_BARE_SYMBOL_STYLES = frozenset({STYLE_VPA, STYLE_TMC}) +_PART_SEPARATOR = {STYLE_TRANSDUCER: r"\mid", STYLE_TMC: r"\mid"} + + +def _render_latex_part(part: EdgePart, style: str) -> str: + if part.kind == PART_KIND: + text = latex_escape(str(part.value)) + return rf"\mathrm{{{text}}}" if style == STYLE_DYCK else text + if part.kind == PART_STACK: + return rf"\uparrow {format_symbol_latex(part.value)}" + if part.kind == PART_MULTIPLICITY: + return rf"\times {part.value}" + if part.kind == PART_MATCH_TAG: + return _format_dyck_match_tag_latex(part.value) + if part.kind in (PART_PROB, PART_QUASIPROB): + return format_prob_latex(part.value) + # symbol / emission / output + symbol = format_symbol_latex(part.value) + return symbol if style in _BARE_SYMBOL_STYLES else rf"\Symbol{{{symbol}}}" + + def _tikz_edge_label(model: StateMachine, transition: Transition) -> str: spec = edge_spec(model, transition) - style = spec.style - - if style == STYLE_SYMBOL_ONLY: - symbol = part_value(spec, PART_SYMBOL) - return format_symbol_only_latex(symbol) if symbol is not None else "" - - if style == STYLE_TRANSDUCER: - symbol = part_value(spec, PART_SYMBOL) - output = part_value(spec, PART_OUTPUT) - if symbol is None and output is None: - return "" - return format_transducer_edge_latex(symbol, output) - - if style == STYLE_PROB_ONLY: - prob = part_value(spec, PART_PROB) - return rf"${format_prob_latex(prob)}$" if prob is not None else "" - - if style == STYLE_EDGE: - label_symbol = part_value(spec, PART_EMISSION, PART_SYMBOL) - value = part_value(spec, PART_PROB, PART_QUASIPROB) - if label_symbol is not None and value is not None: - return format_edge_latex(label_symbol, value) - if label_symbol is not None: - return format_symbol_only_latex(label_symbol) - if value is not None: - return rf"${format_prob_latex(value)}$" + if spec.style in (STYLE_EDGE, STYLE_FALLBACK): + symbol = part_value(spec, PART_EMISSION, PART_SYMBOL) + prob = part_value(spec, PART_PROB, PART_QUASIPROB) + if symbol is not None and prob is not None: + return format_edge_latex(symbol, prob) + parts = [_render_latex_part(part, spec.style) for part in spec.parts] + if not parts: return "" - - if style == STYLE_VPA: - parts: list[str] = [] - for part in spec.parts: - if part.kind == PART_SYMBOL: - parts.append(format_symbol_latex(part.value)) - elif part.kind == PART_KIND: - parts.append(latex_escape(str(part.value))) - elif part.kind == PART_STACK: - parts.append(rf"\uparrow{format_symbol_latex(part.value)}") - return "$" + r"\mid".join(parts) + "$" if parts else "" - - if style == STYLE_DYCK: - parts = [] - for part in spec.parts: - if part.kind == PART_SYMBOL: - parts.append(rf"\Symbol{{{format_symbol_latex(part.value)}}}") - elif part.kind == PART_KIND: - parts.append(rf"\mathrm{{{latex_escape(str(part.value))}}}") - elif part.kind == PART_MATCH_TAG: - parts.append(_format_dyck_match_tag_latex(part.value)) - return "$" + r"\mid ".join(parts) + "$" if parts else "" - - if style == STYLE_TMC: - parts = [] - for part in spec.parts: - if part.kind == PART_SYMBOL: - parts.append(format_symbol_latex(part.value)) - elif part.kind == PART_MULTIPLICITY: - parts.append(latex_escape(f"\\times {part.value}")) - return "$" + r"\mid".join(parts) + "$" if parts else "" - - # STYLE_FALLBACK - label_symbol = part_value(spec, PART_EMISSION, PART_SYMBOL) - prob = part_value(spec, PART_PROB) - if label_symbol is not None and prob is not None: - return format_edge_latex(label_symbol, prob) - if label_symbol is not None: - return format_symbol_only_latex(label_symbol) - if prob is not None: - return rf"${format_prob_latex(prob)}$" - return "" + return "$" + _PART_SEPARATOR.get(spec.style, r"\mid ").join(parts) + "$" def _tikz_display_kwargs(model: StateMachine) -> dict[str, Any]: diff --git a/tests/test_active_learning.py b/tests/test_active_learning.py index 0d58d1e..6bdf3d6 100644 --- a/tests/test_active_learning.py +++ b/tests/test_active_learning.py @@ -6,7 +6,8 @@ import pytest -from sofic.automata.active import ( +from sofic.automata.dfa import DFA +from sofic.automata.learning.active import ( ExhaustiveEquivalenceOracle, FunctionMembershipOracle, LanguageMembershipOracle, @@ -16,7 +17,6 @@ learn_dfa_ttt, learn_mealy_from_transducer, ) -from sofic.automata.dfa import DFA from sofic.automata.transducers import MealyMachine ALPHABET = ("a", "b") @@ -159,7 +159,7 @@ def test_mealy_learns_last_symbol_echo(): def echo(word): return tuple(word) - from sofic.automata.active import FunctionMealyOracle, MealyExhaustiveEquivalenceOracle, learn_mealy_lstar + from sofic.automata.learning.active import FunctionMealyOracle, MealyExhaustiveEquivalenceOracle, learn_mealy_lstar oracle = FunctionMealyOracle(echo) alphabet = ("0", "1") diff --git a/tests/test_atomaton.py b/tests/test_atomaton.py index eacd35d..a720581 100644 --- a/tests/test_atomaton.py +++ b/tests/test_atomaton.py @@ -2,10 +2,10 @@ import pytest -from sofic.automata.atomaton import Atomaton, MaximizedPrimeAtomaton, atomic_states, is_atomic +from sofic.automata.canonical.atomaton import Atomaton, MaximizedPrimeAtomaton, atomic_states, is_atomic +from sofic.automata.canonical.rfsa import CanonicalRFSA from sofic.automata.dfa import DFA from sofic.automata.nfa import NFA -from sofic.automata.rfsa import CanonicalRFSA from sofic.exceptions import SoficValidationError diff --git a/tests/test_bayesian_inference.py b/tests/test_bayesian_inference.py index c3e6c24..da2a9f4 100644 --- a/tests/test_bayesian_inference.py +++ b/tests/test_bayesian_inference.py @@ -5,7 +5,8 @@ import numpy as np import pytest -from sofic.examples.processes import SNS, Even +from sofic.examples import even_process +from sofic.examples.processes import sns from sofic.inference.bayesian import ( BayesianInferenceError, InferEM, @@ -58,22 +59,22 @@ def test_model_comparison_mc_probabilities_normalize(): def test_path_count_em_rejects_nonunifilar_topology(): with pytest.raises(BayesianInferenceError): - PathCountEM(SNS(), list("01")) + PathCountEM(sns(), list("01")) def test_path_count_em_counts_even_process(): - data = list("1111101100") - counts = PathCountEM(Even(), data) + data = [int(symbol) for symbol in "1111101100"] + counts = PathCountEM(even_process(), data) assert counts.get_possible_start_nodes() == ["B"] - assert counts.get_edge_count("B", ("A", "0")) == 3 - assert counts.get_edge_count("B", ("A", "1")) == 3 - assert counts.get_edge_count("B", ("B", "1")) == 4 + assert counts.get_edge_count("B", ("A", 0)) == 3 + assert counts.get_edge_count("B", ("A", 1)) == 3 + assert counts.get_edge_count("B", ("B", 1)) == 4 assert counts.get_node_count("B", "A") == 6 assert counts.get_node_count("B", "B") == 4 def test_infer_em_start_marginalization_and_sample(): - posterior = InferEM(Even(), list("1111101100")) + posterior = InferEM(even_process(), [int(symbol) for symbol in "1111101100"]) assert posterior.start_node_probabilities() == {"B": pytest.approx(1.0)} assert posterior.log_evidence() < 0 start, machine = posterior.generate_sample(rng=np.random.default_rng(0)) diff --git a/tests/test_block_entropy.py b/tests/test_block_entropy.py index 4849f05..b449687 100644 --- a/tests/test_block_entropy.py +++ b/tests/test_block_entropy.py @@ -10,6 +10,7 @@ from hypothesis import given, settings from sofic.examples import fair_coin, golden_mean +from sofic.exceptions import SoficValidationError from sofic.generators.epsilon_machine import EpsilonMachine from sofic.generators.topological_epsilon_enumeration import idfa_string_to_epsilon_machine from sofic.testing.strategies import epsilon_machines @@ -144,7 +145,7 @@ def test_golden_mean_block_entropy_estimates_match_finite_order_values(): def test_block_entropy_estimates_fallback_when_exact_excess_entropy_fails(): - with patch.object(EpsilonMachine, "excess_entropy", side_effect=RuntimeError("no bidirectional")): + with patch.object(EpsilonMachine, "excess_entropy", side_effect=SoficValidationError("no bidirectional")): estimates = golden_mean(0.5).block_entropy_estimates(3, use_exact=True) assert np.isfinite(estimates.excess_entropy) @@ -163,7 +164,7 @@ def test_block_entropy_diagram_uses_exact_excess_entropy_when_available(): def test_block_entropy_diagram_fallback_when_exact_excess_entropy_fails(): - with patch.object(EpsilonMachine, "to_bidirectional", side_effect=RuntimeError("no bidirectional")): + with patch.object(EpsilonMachine, "to_bidirectional", side_effect=SoficValidationError("no bidirectional")): diagram = golden_mean(0.5).block_entropy_diagram(1) assert diagram.excess_entropy == pytest.approx(0.25162916738782304, abs=1e-12) diff --git a/tests/test_buchi.py b/tests/test_buchi.py index 16041d2..0c80703 100644 --- a/tests/test_buchi.py +++ b/tests/test_buchi.py @@ -53,3 +53,8 @@ def test_accepts_omega_non_periodic_raises(): ba = _accepting_loop_ba() with pytest.raises(NotImplementedError): ba.accepts_omega(("a", "b", "a")) + + +def test_empty_loop_is_not_an_omega_word(): + with pytest.raises(ValueError, match="non-empty loop"): + _accepting_loop_ba().accepts_lasso(("a",), ()) diff --git a/tests/test_canonical_rfsa.py b/tests/test_canonical_rfsa.py new file mode 100644 index 0000000..6143a70 --- /dev/null +++ b/tests/test_canonical_rfsa.py @@ -0,0 +1,125 @@ +"""Exact canonical RFSA, maximized prime átomaton, residuals, and atoms.""" + +import pytest +from hypothesis import given, settings + +from sofic.automata.algorithms import equivalent +from sofic.automata.canonical.atomaton import Atomaton, MaximizedPrimeAtomaton, is_atomic +from sofic.automata.canonical.rfsa import CanonicalRFSA, ResidualFiniteStateAutomaton +from sofic.automata.dfa import DFA +from sofic.automata.languages.atoms import atoms, prime_atoms +from sofic.automata.languages.base import AutomatonLanguage +from sofic.automata.languages.residuals import prime_residuals +from sofic.automata.nfa import NFA +from sofic.exceptions import SoficValidationError +from sofic.testing.strategies import dfas + + +def _a_then_n_symbols(n: int) -> NFA: + """NFA for Sigma* a Sigma^n: minimal DFA 2^(n+1) states, canonical RFSA n+2 (Denis et al. 2002).""" + nfa = NFA(input_alphabet=frozenset("ab"), initial_states=frozenset({0}), accepting_states=frozenset({n + 1})) + for state in range(n + 2): + nfa.graph.add_state(state) + nfa.add_transition(0, 0, "a") + nfa.add_transition(0, 0, "b") + nfa.add_transition(0, 1, "a") + for state in range(1, n + 1): + nfa.add_transition(state, state + 1, "a") + nfa.add_transition(state, state + 1, "b") + return nfa + + +@pytest.mark.parametrize("n", [1, 2, 3]) +def test_canonical_rfsa_is_exponentially_smaller_than_minimal_dfa(n): + nfa = _a_then_n_symbols(n) + assert len(list(nfa.minimize().states())) == 2 ** (n + 1) + rfsa = CanonicalRFSA.from_language(nfa) + assert len(list(rfsa.states())) == n + 2 + assert equivalent(rfsa, nfa) + rfsa.validate() + + +def test_prime_residual_count_matches_canonical_rfsa(): + assert len(prime_residuals(AutomatonLanguage(_a_then_n_symbols(1)))) == 3 + + +def test_canonical_rfsa_is_invariant_under_state_renaming(): + dfa = _a_then_n_symbols(2).minimize() + renamed = DFA( + input_alphabet=dfa.input_alphabet, + initial_states=frozenset(("r", s) for s in dfa.initial_states), + accepting_states=frozenset(("r", s) for s in dfa.accepting_states), + ) + for state in dfa.states(): + renamed.graph.add_state(("r", state)) + for t in dfa.transitions(): + renamed.add_transition(("r", t.source), ("r", t.target), t.data["symbol"]) + original = CanonicalRFSA.from_language(dfa) + other = CanonicalRFSA.from_language(renamed) + assert len(list(original.states())) == len(list(other.states())) + assert len(list(original.transitions())) == len(list(other.transitions())) + + +def test_residual_validate_rejects_non_residual_state(): + # State "x" accepts {b}, which is not a residual of L = {ab}. + nfa = ResidualFiniteStateAutomaton( + input_alphabet=frozenset("ab"), initial_states=frozenset({"s"}), accepting_states=frozenset({"f"}) + ) + for state in ("s", "m", "f", "x"): + nfa.graph.add_state(state) + nfa.add_transition("s", "m", "a") + nfa.add_transition("m", "f", "b") + nfa.add_transition("x", "f", "b") + nfa.add_transition("x", "f", "a") + with pytest.raises(SoficValidationError, match="not a residual"): + nfa.validate() + + +@settings(max_examples=60, deadline=None) +@given(dfas(min_states=1, max_states=4)) +def test_canonical_rfsa_and_prime_atomaton_recognize_the_language(dfa): + rfsa = CanonicalRFSA.from_language(dfa) + assert equivalent(rfsa, dfa) + rfsa.validate() + assert len(list(rfsa.states())) <= len(list(dfa.minimize().states())) + + mpa = MaximizedPrimeAtomaton.from_language(dfa) + assert equivalent(mpa, dfa) + mpa.validate() + + +def test_maximized_prime_atomaton_need_not_be_atomic(): + """Its right languages lie between an atom and a maximized atom (Tamm 2015), not on atoms.""" + from sofic.automata.enumeration.icdfa import icdfa_string_to_dfa + + dfa = icdfa_string_to_dfa((0, 1, 0, 2, 0, 1), ("0", "1"), n=3, k=2, final_states=frozenset({0, 1})) + mpa = MaximizedPrimeAtomaton.from_language(dfa) + assert equivalent(mpa, dfa) + assert not is_atomic(mpa) + assert is_atomic(Atomaton.from_language(dfa)) + + +@settings(max_examples=40, deadline=None) +@given(dfas(min_states=1, max_states=4)) +def test_dual_round_trip(dfa): + rfsa = CanonicalRFSA.from_language(dfa) + mpa_of_reverse = rfsa.dual() + assert isinstance(mpa_of_reverse, MaximizedPrimeAtomaton) + assert equivalent(mpa_of_reverse, dfa.reverse()) + back = mpa_of_reverse.dual() + assert isinstance(back, CanonicalRFSA) + assert equivalent(back, dfa) + expected = MaximizedPrimeAtomaton.from_language(dfa.reverse()) + assert len(list(mpa_of_reverse.states())) == len(list(expected.states())) + + +@settings(max_examples=40, deadline=None) +@given(dfas(min_states=1, max_states=4)) +def test_atoms_label_atomaton_states_and_prime_atoms_label_prime_atomaton(dfa): + if not dfa.minimize().accepting_states: + return + language = AutomatonLanguage(dfa) + assert len(atoms(language)) == len(list(Atomaton.from_language(dfa).states())) + assert len(prime_atoms(language)) == len(list(MaximizedPrimeAtomaton.from_language(dfa).states())) + for atom in atoms(language): + assert atom.automaton.minimize().accepting_states diff --git a/tests/test_channel_measures.py b/tests/test_channel_measures.py index fd2b5ca..f40589e 100644 --- a/tests/test_channel_measures.py +++ b/tests/test_channel_measures.py @@ -3,7 +3,7 @@ import pytest from sofic import EpsilonTransducer, MealyHMM -from sofic.examples.processes import BinaryChannel, Delay +from sofic.examples.processes import binary_channel, delay from sofic.generators.channel_measures import ( channel_statistical_complexity, directed_information, @@ -22,34 +22,34 @@ def _iid_input() -> MealyHMM: def test_memoryless_channel_zero_complexity(): - eps = EpsilonTransducer.from_channel(BinaryChannel(0.1, 0.2)) + eps = EpsilonTransducer.from_channel(binary_channel(0.1, 0.2)) assert channel_statistical_complexity(eps, _iid_input()) == pytest.approx(0.0, abs=1e-9) def test_memoryless_channel_zero_transfer_entropy(): - eps = EpsilonTransducer.from_channel(BinaryChannel(0.1, 0.2)) + eps = EpsilonTransducer.from_channel(binary_channel(0.1, 0.2)) assert transfer_entropy(eps, _iid_input()) == pytest.approx(0.0, abs=1e-9) def test_memory_channel_positive_complexity(): - eps = EpsilonTransducer.from_channel(Delay(1)) + eps = EpsilonTransducer.from_channel(delay(1)) assert channel_statistical_complexity(eps, _iid_input()) > 0.5 def test_driven_entropy_rate_bounds(): - eps = EpsilonTransducer.from_channel(BinaryChannel(0.1, 0.2)) + eps = EpsilonTransducer.from_channel(binary_channel(0.1, 0.2)) rate = driven_entropy_rate(eps, _iid_input()) assert 0.0 <= rate <= 1.0 + 1e-9 def test_directed_information_positive(): - eps = EpsilonTransducer.from_channel(BinaryChannel(0.1, 0.2)) + eps = EpsilonTransducer.from_channel(binary_channel(0.1, 0.2)) assert directed_information(eps, _iid_input(), length=2) > 0.0 def test_complete_skips_unused_reject_sink(): """Already-total channels must not gain an unused absorbing ``?`` state.""" - eps = EpsilonTransducer.from_channel(BinaryChannel(0.1, 0.2)) + eps = EpsilonTransducer.from_channel(binary_channel(0.1, 0.2)) completed = eps.complete(frozenset({"0", "1"})) assert "?" not in completed.states() assert list(completed.states()) == list(eps.states()) @@ -58,24 +58,24 @@ def test_complete_skips_unused_reject_sink(): def test_directed_information_ignores_unreachable_error_class(): """Reducible joints with an unused error sink must keep positive DI. - ``compose_tg(..., complete=True)`` used to always attach an absorbing ``?`` + ``compose_transducer_generator(..., complete=True)`` used to always attach an absorbing ``?`` component; the eigenvector stationary law is then non-unique and can put all mass on ``?`` (PYTHONHASHSEED-dependent), zeroing directed information. """ - from sofic.automata.transducer_operations import compose_tg + from sofic.automata.transducer_operations import compose_transducer_generator from sofic.generators.directional_flow import directed_information as di_flow - from sofic.generators.hmm_inference import _stationary_emission_tensors + from sofic.generators.matrices import emission_tensors - eps = EpsilonTransducer.from_channel(BinaryChannel(0.1, 0.2)) + eps = EpsilonTransducer.from_channel(binary_channel(0.1, 0.2)) # Force the historical reducible joint even after complete() stops adding an # unused sink: compose with an explicit completed copy that includes ``?``. completed = eps.copy() completed.graph.add_state("?") completed.add_transition("?", "?", "0", "?", prob=1.0) completed.add_transition("?", "?", "1", "?", prob=1.0) - joint = compose_tg(completed, _iid_input(), joint=True, complete=False) + joint = compose_transducer_generator(completed, _iid_input(), joint=True, complete=False) assert ("S", "?") in list(joint.states()) - pi, _tensors = _stationary_emission_tensors(joint) + pi, _tensors = emission_tensors(joint, policy="stationary") idx = joint.reindex() error_index = idx.index(("S", "?")) assert pi[error_index] == pytest.approx(0.0, abs=1e-12) @@ -83,7 +83,7 @@ def test_directed_information_ignores_unreachable_error_class(): def test_method_dispatch_matches_functions(): - eps = EpsilonTransducer.from_channel(BinaryChannel(0.1, 0.2)) + eps = EpsilonTransducer.from_channel(binary_channel(0.1, 0.2)) inp = _iid_input() assert eps.statistical_complexity(inp) == pytest.approx(channel_statistical_complexity(eps, inp)) assert eps.transfer_entropy(inp) == pytest.approx(transfer_entropy(eps, inp)) diff --git a/tests/test_correctness_regressions.py b/tests/test_correctness_regressions.py new file mode 100644 index 0000000..4b3e040 --- /dev/null +++ b/tests/test_correctness_regressions.py @@ -0,0 +1,108 @@ +"""Regression tests for correctness fixes found in the package review.""" + +import numpy as np +import pytest + +from sofic.automata.algorithms import equivalent +from sofic.automata.dfa import DFA +from sofic.examples import even_process, golden_mean +from sofic.exceptions import SoficValidationError +from sofic.generators.markov import MarkovChain +from sofic.generators.mealy import MealyHMM +from sofic.graph import ATTR_MULTIPLICITY, ATTR_SYMBOL +from sofic.shifts.sft import ShiftOfFiniteType +from sofic.shifts.tmc import TopologicalMarkovChain + + +def test_tmc_multiplicity_counts_in_entropy_and_parry_measure(): + tmc = TopologicalMarkovChain(symbol_alphabet=frozenset({"a"})) + tmc.graph.add_state("s") + tmc.graph.add_transition("s", "s", **{ATTR_SYMBOL: "a", ATTR_MULTIPLICITY: 2}) + assert tmc.topological_entropy() == pytest.approx(1.0) + + matrix_tmc = TopologicalMarkovChain.from_adjacency(np.array([[2, 1], [1, 0]]), symbol_alphabet=frozenset("ab")) + expected = np.log2(max(abs(np.linalg.eigvals(np.array([[2, 1], [1, 0]]))))) + assert matrix_tmc.topological_entropy() == pytest.approx(expected) + parry = matrix_tmc.parry_measure() + parry.validate() + weights = {} + for t in parry.transitions(): + weights[(t.source, t.target)] = weights.get((t.source, t.target), 0.0) + t.data["prob"] + for state in parry.states(): + assert sum(w for (s, _), w in weights.items() if s == state) == pytest.approx(1.0) + + +def test_golden_mean_topological_entropy_is_log2_golden_ratio(): + tmc = TopologicalMarkovChain.from_adjacency(np.array([[1, 1], [1, 0]]), symbol_alphabet=frozenset("01")) + assert tmc.topological_entropy() == pytest.approx(np.log2((1 + np.sqrt(5)) / 2)) + + +def test_sft_construction_raises_instead_of_truncating(): + forbidden = {tuple("0" * 9)} + with pytest.raises(SoficValidationError, match="max_states"): + ShiftOfFiniteType.from_forbidden_words(forbidden, frozenset("01"), max_states=8) + + +def test_markov_words_of_length_zero_respects_initial_distribution(): + chain = MarkovChain(initial_distribution={"a": 1.0}) + chain.graph.add_state("a") + chain.graph.add_state("b") + chain.add_transition("a", "b", 1.0) + chain.add_transition("b", "a", 1.0) + assert chain.words_of_length(0) == {(): pytest.approx(1.0)} + assert chain.words_of_length(2) == {("a", "b"): pytest.approx(1.0)} + + stationary = MarkovChain() + stationary.graph.add_state("a") + stationary.graph.add_state("b") + stationary.add_transition("a", "b", 1.0) + stationary.add_transition("b", "a", 1.0) + words = stationary.words_of_length(1) + assert words == {("a",): pytest.approx(0.5), ("b",): pytest.approx(0.5)} + + +def test_markov_sample_path_starts_from_initial_distribution(): + chain = MarkovChain(initial_distribution={"b": 1.0}) + chain.graph.add_state("a") + chain.graph.add_state("b") + chain.add_transition("a", "b", 1.0) + chain.add_transition("b", "a", 1.0) + for seed in range(5): + assert chain.sample_path(3, rng=np.random.default_rng(seed)) == ["b", "a", "b"] + + +def test_sample_without_initial_mass_raises(): + hmm = MealyHMM(initial_distribution={"A": 0.0}, observation_alphabet=frozenset({0})) + hmm.graph.add_state("A") + hmm.add_transition("A", "A", 0, 1.0) + with pytest.raises(ValueError, match="no mass"): + hmm.sample(3, rng=np.random.default_rng(0)) + + +def test_block_entropy_estimates_exact_crypticity_is_cmu_minus_excess_entropy(): + machine = even_process() + estimates = machine.block_entropy_estimates(3, use_exact=True) + assert estimates.crypticity == pytest.approx(estimates.statistical_complexity - estimates.excess_entropy) + assert estimates.crypticity == pytest.approx(machine.crypticity(), abs=1e-9) + + +def test_log_likelihood_is_in_bits(): + machine = golden_mean(0.5) + observations = [0, 0, 0, 0] + assert 2.0 ** machine.log_likelihood(observations) == pytest.approx(machine.word_probability(observations)) + + +def _single_symbol_dfa(symbols: set[str]) -> DFA: + dfa = DFA(input_alphabet=frozenset(symbols), initial_states=frozenset({0}), accepting_states=frozenset({1})) + dfa.graph.add_state(0) + dfa.graph.add_state(1) + for symbol in symbols: + dfa.add_transition(0, 1, symbol) + return dfa + + +def test_equivalent_does_not_hide_differences_outside_the_given_alphabet(): + only_a = _single_symbol_dfa({"a"}) + a_or_b = _single_symbol_dfa({"a", "b"}) + assert not equivalent(only_a, a_or_b, frozenset({"a"})) + assert equivalent(only_a, _single_symbol_dfa({"a"})) diff --git a/tests/test_covers.py b/tests/test_covers.py index 9172871..20d2044 100644 --- a/tests/test_covers.py +++ b/tests/test_covers.py @@ -1,30 +1,104 @@ """Tests for Fischer and Krieger covers.""" +import networkx as nx import pytest +from sofic.exceptions import SoficValidationError from sofic.graph import ATTR_SYMBOL from sofic.shifts.covers import LeftFischerCover, LeftKriegerCover, RightFischerCover, RightKriegerCover from sofic.shifts.sofic import SoficShift -def _golden_mean() -> SoficShift: - shift = SoficShift(symbol_alphabet=frozenset({"0", "1"})) - shift.graph.add_state("A") - shift.graph.add_state("B") - shift.graph.add_transition("A", "B", **{ATTR_SYMBOL: "1"}) - shift.graph.add_transition("B", "A", **{ATTR_SYMBOL: "0"}) - shift.graph.add_transition("B", "B", **{ATTR_SYMBOL: "1"}) +def _shift(edges, alphabet=("0", "1")) -> SoficShift: + shift = SoficShift(symbol_alphabet=frozenset(alphabet)) + for source, target, symbol in edges: + shift.graph.add_state(source) + shift.graph.add_state(target) + shift.graph.add_transition(source, target, **{ATTR_SYMBOL: symbol}) return shift -@pytest.mark.parametrize("cls", [LeftFischerCover, RightFischerCover]) -def test_fischer_cover_from_sofic(cls): - cover = cls.from_sofic(_golden_mean()) +def _golden_mean() -> SoficShift: + return _shift([("A", "B", "1"), ("B", "A", "0"), ("B", "B", "0"), ("A", "A", "0")]) + + +def _even_shift() -> SoficShift: + """Runs of 1s between 0s have even length; Krieger cover has three vertices.""" + return _shift([("A", "A", "0"), ("A", "B", "1"), ("B", "A", "1")]) + + +def _nondeterministic_even_shift() -> SoficShift: + # Two copies of the even-shift presentation glued nondeterministically. + return _shift( + [ + ("A", "A", "0"), + ("A", "B", "1"), + ("B", "A", "1"), + ("A", "C", "0"), + ("C", "D", "1"), + ("D", "C", "1"), + ("C", "A", "0"), + ] + ) + + +def _language(shift: SoficShift, max_length: int = 8) -> set[tuple]: + return {word for n in range(max_length + 1) for word in shift.factor_language(n)} + + +def _is_left_resolving(shift: SoficShift) -> bool: + return SoficShift(graph=shift.graph.reverse(), symbol_alphabet=shift.symbol_alphabet).is_unifilar() + + +@pytest.mark.parametrize("builder", [_golden_mean, _even_shift, _nondeterministic_even_shift]) +def test_right_fischer_cover_is_minimal_right_resolving_and_presents_shift(builder): + shift = builder() + cover = RightFischerCover.from_presentation(shift) cover.validate() - assert len(list(cover.states())) >= 1 + assert cover.is_unifilar() + assert nx.is_strongly_connected(cover.graph.nx) + assert _language(cover) == _language(shift) + assert len(list(cover.states())) == 2 + + +@pytest.mark.parametrize("builder", [_golden_mean, _even_shift, _nondeterministic_even_shift]) +def test_left_fischer_cover_is_left_resolving_and_presents_shift(builder): + shift = builder() + cover = LeftFischerCover.from_presentation(shift) + assert _is_left_resolving(cover) + assert _language(cover) == _language(shift) + + +def test_even_shift_krieger_cover_has_three_vertices_and_contains_fischer_cover(): + shift = _even_shift() + krieger = RightKriegerCover.from_presentation(shift) + fischer = RightFischerCover.from_presentation(shift) + assert krieger.is_unifilar() + assert len(list(krieger.states())) == 3 + assert _language(krieger) == _language(shift) + condensation = nx.condensation(krieger.graph.nx) + terminal = [n for n in condensation.nodes if condensation.out_degree(n) == 0] + assert len(terminal) == 1 + assert len(condensation.nodes[terminal[0]]["members"]) == len(list(fischer.states())) + + +def test_golden_mean_krieger_cover_equals_fischer_cover_size(): + shift = _golden_mean() + assert len(list(RightKriegerCover.from_presentation(shift).states())) == 2 + assert len(list(LeftKriegerCover.from_presentation(shift).states())) == 2 + + +@pytest.mark.parametrize("builder", [_golden_mean, _even_shift, _nondeterministic_even_shift]) +def test_left_krieger_cover_is_left_resolving_and_presents_shift(builder): + shift = builder() + cover = LeftKriegerCover.from_presentation(shift) + assert _is_left_resolving(cover) + assert _language(cover) == _language(shift) -@pytest.mark.parametrize("cls", [LeftKriegerCover, RightKriegerCover]) -def test_krieger_cover_not_implemented(cls): - with pytest.raises(NotImplementedError, match="Krieger"): - cls.from_sofic(_golden_mean()) +def test_fischer_cover_rejects_reducible_shift(): + reducible = _shift([("A", "A", "0"), ("B", "B", "1")]) + with pytest.raises(SoficValidationError, match="irreducible"): + RightFischerCover.from_presentation(reducible) + krieger = RightKriegerCover.from_presentation(reducible) + assert _language(krieger) == _language(reducible) diff --git a/tests/test_edge_machine.py b/tests/test_edge_machine.py index 62da4e5..9f3eecf 100644 --- a/tests/test_edge_machine.py +++ b/tests/test_edge_machine.py @@ -71,14 +71,14 @@ def test_edge_machine_preserves_block_distribution(builder): hmm = builder() edge = hmm.to_edge_machine() symbols = sorted(hmm.observation_alphabet, key=repr) - dist_hmm0 = hmm.joint_block_distribution(history_length=0) - dist_edge0 = edge.joint_block_distribution(history_length=0) + dist_hmm0 = hmm.joint_block_distribution(block_length=1) + dist_edge0 = edge.joint_block_distribution(block_length=1) for symbol in symbols: outcome = (symbol,) assert dist_hmm0[outcome] == pytest.approx(dist_edge0[outcome], abs=1e-9) - dist_hmm1 = hmm.joint_block_distribution(history_length=1) - dist_edge1 = edge.joint_block_distribution(history_length=1) + dist_hmm1 = hmm.joint_block_distribution(block_length=2) + dist_edge1 = edge.joint_block_distribution(block_length=2) for past in symbols: for present in symbols: outcome = (past, present) diff --git a/tests/test_epsilon_inference.py b/tests/test_epsilon_inference.py index 532b49d..66a49dc 100644 --- a/tests/test_epsilon_inference.py +++ b/tests/test_epsilon_inference.py @@ -10,10 +10,11 @@ import pytest from sofic.examples.epsilon_machines import bernoulli, even_process, golden_mean -from sofic.generators.epsilon_inference import cssr, spectral, subtree_merge from sofic.generators.epsilon_machine import EpsilonMachine -from sofic.generators.hmm_inference import sample +from sofic.generators.sampling import sample from sofic.graph import ATTR_EMISSION, ATTR_PROB +from sofic.inference.cssr import learn_epsilon_machine_cssr, learn_epsilon_machine_subtree +from sofic.inference.spectral import learn_epsilon_machine_spectral def _transition_signature(hmm: EpsilonMachine) -> dict[Hashable, tuple[tuple[Any, Hashable, float], ...]]: @@ -85,7 +86,7 @@ def rng() -> np.random.Generator: def test_cssr_bernoulli_single_state(rng: np.random.Generator): oracle = bernoulli(0.3) observations, _ = sample(oracle, 500, rng) - inferred = cssr(observations, alpha=0.01) + inferred = learn_epsilon_machine_cssr(observations, alpha=0.01) inferred.validate() assert len(list(inferred.states())) == 1 @@ -101,7 +102,7 @@ def test_from_sequence_cssr_dispatch(rng: np.random.Generator): def test_cssr_golden_mean_recovers_two_states(rng: np.random.Generator): oracle = golden_mean(0.5) observations, _ = sample(oracle, 8000, rng) - inferred = cssr(observations, Lmax=3, alpha=0.001) + inferred = learn_epsilon_machine_cssr(observations, max_history=3, alpha=0.001) inferred.validate() assert len(list(inferred.states())) == 2 assert _signatures_isomorphic(inferred, oracle, prob_tol=0.1) @@ -110,7 +111,7 @@ def test_cssr_golden_mean_recovers_two_states(rng: np.random.Generator): def test_cssr_even_process_recovers_two_states(rng: np.random.Generator): oracle = even_process(0.5) observations, _ = sample(oracle, 12000, rng) - inferred = cssr(observations, Lmax=4, alpha=0.001) + inferred = learn_epsilon_machine_cssr(observations, max_history=4, alpha=0.001) inferred.validate() assert len(list(inferred.states())) == 2 assert _signatures_isomorphic(inferred, oracle, prob_tol=0.12) @@ -119,7 +120,7 @@ def test_cssr_even_process_recovers_two_states(rng: np.random.Generator): def test_subtree_merge_golden_mean(rng: np.random.Generator): oracle = golden_mean(0.5) observations, _ = sample(oracle, 10000, rng) - inferred = subtree_merge(observations, L=2, delta=0.0) + inferred = learn_epsilon_machine_subtree(observations, max_history=2, delta=0.0) inferred.validate() assert len(list(inferred.states())) == 2 assert _signatures_isomorphic(inferred, oracle, prob_tol=0.12) @@ -128,7 +129,7 @@ def test_subtree_merge_golden_mean(rng: np.random.Generator): def test_from_sequence_subtree_dispatch(rng: np.random.Generator): oracle = golden_mean(0.5) observations, _ = sample(oracle, 6000, rng) - inferred = EpsilonMachine.from_sequence(observations, method="subtree", L=2) + inferred = EpsilonMachine.from_sequence(observations, method="subtree", max_history=2) inferred.validate() assert len(list(inferred.states())) >= 2 @@ -142,7 +143,7 @@ def test_cssr_initial_distribution_matches_occupation(rng: np.random.Generator): """ oracle = golden_mean(0.5) observations, _ = sample(oracle, 8000, rng) - inferred = cssr(observations, Lmax=3, alpha=0.001) + inferred = learn_epsilon_machine_cssr(observations, max_history=3, alpha=0.001) inferred.validate() idx = inferred.reindex() @@ -154,18 +155,20 @@ def test_cssr_initial_distribution_matches_occupation(rng: np.random.Generator): def test_cssr_short_sequence_raises(): with pytest.raises(ValueError, match="at least two"): - cssr([0]) + learn_epsilon_machine_cssr([0]) def test_subtree_merge_short_sequence_raises(): with pytest.raises(ValueError, match="at least two"): - subtree_merge([1], L=1) + learn_epsilon_machine_subtree([1], max_history=1) def test_spectral_bernoulli_single_state(): oracle = bernoulli(0.3) alphabet = sorted(oracle.observation_alphabet, key=repr) - inferred = spectral(word_probability=oracle.word_probability, alphabet=alphabet, prefix_length=2, rank=1) + inferred = learn_epsilon_machine_spectral( + word_probability=oracle.word_probability, alphabet=alphabet, prefix_length=2, rank=1 + ) inferred.validate() assert len(list(inferred.states())) == 1 assert inferred.entropy_rate() == pytest.approx(oracle.entropy_rate(), abs=1e-9) @@ -174,7 +177,9 @@ def test_spectral_bernoulli_single_state(): def test_spectral_golden_mean_recovers_two_states(): oracle = golden_mean(0.5) alphabet = sorted(oracle.observation_alphabet, key=repr) - inferred = spectral(word_probability=oracle.word_probability, alphabet=alphabet, prefix_length=3, rank=2) + inferred = learn_epsilon_machine_spectral( + word_probability=oracle.word_probability, alphabet=alphabet, prefix_length=3, rank=2 + ) inferred.validate() assert len(list(inferred.states())) == 2 assert inferred.entropy_rate() == pytest.approx(oracle.entropy_rate(), abs=1e-6) @@ -185,7 +190,9 @@ def test_spectral_golden_mean_recovers_two_states(): def test_spectral_even_process_recovers_two_states(): oracle = even_process(0.5) alphabet = sorted(oracle.observation_alphabet, key=repr) - inferred = spectral(word_probability=oracle.word_probability, alphabet=alphabet, prefix_length=3, rank=2) + inferred = learn_epsilon_machine_spectral( + word_probability=oracle.word_probability, alphabet=alphabet, prefix_length=3, rank=2 + ) inferred.validate() assert len(list(inferred.states())) == 2 assert inferred.entropy_rate() == pytest.approx(oracle.entropy_rate(), abs=1e-6) @@ -207,17 +214,17 @@ def test_from_sequence_unknown_method(): @pytest.mark.parametrize( - ("name", "Lmax"), - [("Even", 3), ("Even", 5), ("GoldenMean", 3), ("Nemo", 4), ("RkGM", 5)], + ("name", "max_history"), + [("even_process", 3), ("even_process", 5), ("golden_mean_forbid_00", 3), ("nemo_process", 4), ("rk_gm", 5)], ) -def test_cssr_recovers_synchronizable_processes(name: str, Lmax: int): +def test_cssr_recovers_synchronizable_processes(name: str, max_history: int): """Regression: appended (not prepended) suffixes and untruncated successors dropped edges, so these raised StochasticValidationError or returned h_mu = 0.""" - from sofic.examples import processes + from sofic import examples - oracle = processes.RkGM(5, 3) if name == "RkGM" else getattr(processes, name)() + oracle = examples.rk_gm(5, 3) if name == "rk_gm" else getattr(examples, name)() observations, _ = sample(oracle, 20000, np.random.default_rng(5)) - inferred = cssr(observations, Lmax=Lmax, alpha=0.001) + inferred = learn_epsilon_machine_cssr(observations, max_history=max_history, alpha=0.001) inferred.validate() assert inferred.is_unifilar() assert len(list(inferred.states())) == len(list(oracle.states())) @@ -225,51 +232,53 @@ def test_cssr_recovers_synchronizable_processes(name: str, Lmax: int): def test_cssr_even_process_ignores_truncated_nonsynchronizing_suffix(): - """At Lmax = 3 the successor of ``011`` on ``1`` truncates to the ambiguous ``111``.""" + """At max_history = 3 the successor of ``011`` on ``1`` truncates to the ambiguous ``111``.""" oracle = even_process(0.5) observations, _ = sample(oracle, 20000, np.random.default_rng(5)) - inferred = cssr(observations, Lmax=3, alpha=0.01) + inferred = learn_epsilon_machine_cssr(observations, max_history=3, alpha=0.01) assert _signatures_isomorphic(inferred, oracle, prob_tol=0.03) def test_cssr_default_lmax_does_not_oversplit(): for oracle, n_states in [(even_process(0.5), 2), (bernoulli(0.3), 1)]: observations, _ = sample(oracle, 20000, np.random.default_rng(7)) - inferred = cssr(observations) + inferred = learn_epsilon_machine_cssr(observations) inferred.validate() assert len(list(inferred.states())) == n_states def test_cssr_short_lmax_still_emits_every_symbol(): - """Lmax below the Markov order cannot recover RkGM(5, 3), but must not collapse to a trap state.""" + """max_history below the Markov order cannot recover rk_gm(5, 3), but must not collapse to a trap state.""" from sofic.examples import processes - observations, _ = sample(processes.RkGM(5, 3), 20000, np.random.default_rng(5)) - inferred = cssr(observations, Lmax=3, alpha=0.001) + observations, _ = sample(processes.rk_gm(5, 3), 20000, np.random.default_rng(5)) + inferred = learn_epsilon_machine_cssr(observations, max_history=3, alpha=0.001) inferred.validate() assert {t.data[ATTR_EMISSION] for t in inferred.transitions()} == {"0", "1"} assert inferred.entropy_rate() > 0.0 def test_cssr_non_synchronizable_process_returns_valid_machine(): - from sofic.examples import processes + from sofic.examples import alternating_biased_coins - oracle = processes.ABC() + oracle = alternating_biased_coins(0.25, 0.75) observations, _ = sample(oracle, 20000, np.random.default_rng(5)) - inferred = cssr(observations, Lmax=4, alpha=0.001) + inferred = learn_epsilon_machine_cssr(observations, max_history=4, alpha=0.001) inferred.validate() assert inferred.entropy_rate() >= oracle.entropy_rate() - 0.02 -@pytest.mark.parametrize(("name", "L", "n_states"), [("Even", 3, 2), ("GoldenMean", 2, 2), ("RkGM", 5, 8)]) +@pytest.mark.parametrize( + ("name", "L", "n_states"), [("even_process", 3, 2), ("golden_mean_forbid_00", 2, 2), ("rk_gm", 5, 8)] +) def test_subtree_merge_default_delta_recovers_process(name: str, L: int, n_states: int): """Regression: the default delta = 0 compared sampled morphs to within 1e-3 and successors were never truncated, so this raised StochasticValidationError.""" - from sofic.examples import processes + from sofic import examples - oracle = processes.RkGM(5, 3) if name == "RkGM" else getattr(processes, name)() + oracle = examples.rk_gm(5, 3) if name == "rk_gm" else getattr(examples, name)() observations, _ = sample(oracle, 20000, np.random.default_rng(5)) - inferred = subtree_merge(observations, L=L) + inferred = learn_epsilon_machine_subtree(observations, max_history=L) inferred.validate() assert len(list(inferred.states())) == n_states assert inferred.entropy_rate() == pytest.approx(oracle.entropy_rate(), abs=0.02) @@ -289,35 +298,35 @@ def _has_markov_order_selection() -> bool: @needs_markov_order def test_suggest_lmax_markov_sources(): from sofic.examples import processes - from sofic.generators.epsilon_inference import suggest_lmax + from sofic.inference.cssr import suggest_max_history observations, _ = sample(golden_mean(0.5), 4000, np.random.default_rng(1)) - assert suggest_lmax(observations) == 1 - observations, _ = sample(processes.RkGM(3, 2), 20000, np.random.default_rng(2)) - assert suggest_lmax(observations, method="bic") == 3 + assert suggest_max_history(observations) == 1 + observations, _ = sample(processes.rk_gm(3, 2), 20000, np.random.default_rng(2)) + assert suggest_max_history(observations, method="bic") == 3 @needs_markov_order def test_suggest_lmax_grows_for_even_process(): """The even process has infinite Markov order, so the suggestion grows with data.""" - from sofic.generators.epsilon_inference import suggest_lmax + from sofic.inference.cssr import suggest_max_history short, _ = sample(even_process(0.5), 300, np.random.default_rng(3)) long, _ = sample(even_process(0.5), 30000, np.random.default_rng(3)) - assert suggest_lmax(long, method="bic") > suggest_lmax(short, method="bic") + assert suggest_max_history(long, method="bic") > suggest_max_history(short, method="bic") @needs_markov_order def test_cssr_auto_lmax_golden_mean(rng: np.random.Generator): observations, _ = sample(golden_mean(0.5), 8000, rng) - inferred = cssr(observations, Lmax="auto", alpha=0.001) + inferred = learn_epsilon_machine_cssr(observations, max_history="auto", alpha=0.001) assert len(list(inferred.states())) == 2 assert _signatures_isomorphic(inferred, golden_mean(0.5), prob_tol=0.1) def test_exact_morph_test_small_counts(): """With tiny counts the exact test is calibrated where the chi-squared limit is not.""" - from sofic.generators.epsilon_inference import SuffixCounts, morphs_differ + from sofic.inference.cssr import SuffixCounts, morphs_differ rng = np.random.default_rng(4) rejections = {"g": 0, "exact": 0} @@ -336,31 +345,31 @@ def test_exact_morph_test_small_counts(): def test_exact_morph_test_is_deterministic(rng: np.random.Generator): observations, _ = sample(golden_mean(0.5), 3000, rng) - first = cssr(observations, Lmax=3, alpha=0.01, test="exact") - second = cssr(observations, Lmax=3, alpha=0.01, test="exact") + first = learn_epsilon_machine_cssr(observations, max_history=3, alpha=0.01, test="exact") + second = learn_epsilon_machine_cssr(observations, max_history=3, alpha=0.01, test="exact") assert _transition_signature(first) == _transition_signature(second) assert len(list(first.states())) == 2 def test_cssr_bonferroni_reduces_spurious_states(): - """An i.i.d. source with a long Lmax: the corrected test keeps a single state.""" + """An i.i.d. source with a long max_history: the corrected test keeps a single state.""" observations, _ = sample(bernoulli(0.3), 3000, np.random.default_rng(6)) - inferred = cssr(observations, Lmax=6, alpha=0.05, correction="bonferroni") + inferred = learn_epsilon_machine_cssr(observations, max_history=6, alpha=0.05, correction="bonferroni") assert len(list(inferred.states())) == 1 with pytest.raises(ValueError, match="unknown correction"): - cssr(observations, Lmax=2, correction="holm") + learn_epsilon_machine_cssr(observations, max_history=2, correction="holm") @pytest.mark.parametrize("kwargs", [{"test": "exact"}, {"correction": "bonferroni"}, {"alpha": 0.001}]) def test_subtree_merge_options_golden_mean(kwargs): observations, _ = sample(golden_mean(0.5), 6000, np.random.default_rng(7)) - inferred = subtree_merge(observations, L=2, **kwargs) + inferred = learn_epsilon_machine_subtree(observations, max_history=2, **kwargs) assert len(list(inferred.states())) == 2 with pytest.raises(ValueError, match="unknown correction"): - subtree_merge(observations, L=2, correction="holm") + learn_epsilon_machine_subtree(observations, max_history=2, correction="holm") @needs_markov_order def test_subtree_merge_auto_depth(): observations, _ = sample(golden_mean(0.5), 6000, np.random.default_rng(8)) - assert len(list(subtree_merge(observations, L="auto").states())) == 2 + assert len(list(learn_epsilon_machine_subtree(observations, max_history="auto").states())) == 2 diff --git a/tests/test_epsilon_transducer.py b/tests/test_epsilon_transducer.py index 9d7ce8b..0a82d7f 100644 --- a/tests/test_epsilon_transducer.py +++ b/tests/test_epsilon_transducer.py @@ -3,8 +3,8 @@ import pytest from sofic import EpsilonTransducer, MealyHMM -from sofic.automata.transducer_operations import compose_tg -from sofic.examples.processes import GME, RCT, BinaryChannel, GMtoEven +from sofic.automata.transducer_operations import compose_transducer_generator +from sofic.examples.processes import binary_channel, gm_to_even, gme, rct from sofic.exceptions import StochasticValidationError, UnifilarityError from sofic.graph import ATTR_OUTPUT, ATTR_PROB, ATTR_SYMBOL @@ -21,10 +21,10 @@ def _iid_input() -> MealyHMM: @pytest.mark.parametrize( ("channel", "expected_states"), [ - (BinaryChannel(0.1, 0.2), 1), - (GMtoEven(), 2), - (RCT(0.5), 3), - (GME(), 2), + (binary_channel(0.1, 0.2), 1), + (gm_to_even(), 2), + (rct(0.5), 3), + (gme(), 2), ], ) def test_from_channel_minimizes(channel, expected_states): @@ -35,7 +35,7 @@ def test_from_channel_minimizes(channel, expected_states): def test_memoryless_channel_is_single_causal_state(): - eps = EpsilonTransducer.from_channel(BinaryChannel(0.1, 0.2)) + eps = EpsilonTransducer.from_channel(binary_channel(0.1, 0.2)) assert len(eps.causal_states()) == 1 rows = {} for transition in eps.transitions(): @@ -48,7 +48,7 @@ def test_memoryless_channel_is_single_causal_state(): def test_initial_distribution_normalized(): - eps = EpsilonTransducer.from_channel(GMtoEven()) + eps = EpsilonTransducer.from_channel(gm_to_even()) assert sum(eps.initial_distribution.values()) == pytest.approx(1.0) assert set(eps.initial_distribution) <= set(eps.states()) @@ -79,7 +79,7 @@ def _non_unifilar_channel(): def test_validate_rejects_bad_initial_distribution(): - eps = EpsilonTransducer.from_channel(BinaryChannel(0.1, 0.2)) + eps = EpsilonTransducer.from_channel(binary_channel(0.1, 0.2)) eps.initial_distribution = {next(iter(eps.states())): 0.5} with pytest.raises(StochasticValidationError): eps.validate() @@ -103,7 +103,7 @@ def test_validate_rejects_non_unifilar_direct(): def test_from_joint_generator_recovers_memoryless(): - joint = compose_tg(BinaryChannel(0.1, 0.2), _iid_input(), joint=True) + joint = compose_transducer_generator(binary_channel(0.1, 0.2), _iid_input(), joint=True) eps = EpsilonTransducer.from_joint_generator(joint) eps.validate() assert len(list(eps.states())) == 1 @@ -111,13 +111,13 @@ def test_from_joint_generator_recovers_memoryless(): def test_from_iohmm_alias(): - eps = EpsilonTransducer.from_iohmm(GMtoEven()) + eps = EpsilonTransducer.from_iohmm(gm_to_even()) assert isinstance(eps, EpsilonTransducer) assert len(list(eps.states())) == 2 def test_yaml_round_trip(): - eps = EpsilonTransducer.from_channel(RCT(0.5)) + eps = EpsilonTransducer.from_channel(rct(0.5)) restored = EpsilonTransducer.from_yaml(eps.to_yaml()) assert isinstance(restored, EpsilonTransducer) assert len(list(restored.states())) == len(list(eps.states())) @@ -125,7 +125,7 @@ def test_yaml_round_trip(): def test_wfst_round_trip_preserves_structure(): - eps = EpsilonTransducer.from_channel(GMtoEven()) + eps = EpsilonTransducer.from_channel(gm_to_even()) wfst = eps.to_wfst() recovered = EpsilonTransducer.from_wfst(wfst) assert len(list(recovered.states())) == len(list(eps.states())) diff --git a/tests/test_epsilon_transducer_inference.py b/tests/test_epsilon_transducer_inference.py index 6e7137f..9a84939 100644 --- a/tests/test_epsilon_transducer_inference.py +++ b/tests/test_epsilon_transducer_inference.py @@ -2,11 +2,14 @@ import numpy as np import pytest +from hypothesis import given, settings +from hypothesis import strategies as st +from hypothesis.extra import numpy as hnp from sofic import EpsilonTransducer, MealyHMM -from sofic.automata.transducer_operations import compose_tg -from sofic.examples.processes import BinaryChannel, Delay -from sofic.generators.epsilon_transducer_inference import JointSuffixCounts, transcssr +from sofic.automata.transducer_operations import compose_transducer_generator +from sofic.examples.processes import binary_channel, delay +from sofic.inference.cssr import JointSuffixCounts, learn_epsilon_transducer_cssr def _iid_input() -> MealyHMM: @@ -19,7 +22,7 @@ def _iid_input() -> MealyHMM: def _paired_samples(channel, n, seed): - joint = compose_tg(channel, _iid_input(), joint=True) + joint = compose_transducer_generator(channel, _iid_input(), joint=True) observations, _ = joint.sample(n, np.random.default_rng(seed)) xs = [pair[0] for pair in observations] ys = [pair[1] for pair in observations] @@ -27,7 +30,7 @@ def _paired_samples(channel, n, seed): def _memoryless_reconstruction(n: int = 20000, *, max_seeds: int = 8) -> EpsilonTransducer: - """Recover a single-state ε-transducer for ``BinaryChannel(0.1, 0.2)``. + """Recover a single-state ε-transducer for ``binary_channel(0.1, 0.2)``. CSSR's χ² split decision is float-sensitive across platforms, so a fixed ``(n, seed)`` can over-split on some runners. Cap history depth at 1 (enough @@ -35,13 +38,13 @@ def _memoryless_reconstruction(n: int = 20000, *, max_seeds: int = 8) -> Epsilon single-state. """ for seed in range(max_seeds): - xs, ys = _paired_samples(BinaryChannel(0.1, 0.2), n, seed=seed) - candidate = transcssr( + xs, ys = _paired_samples(binary_channel(0.1, 0.2), n, seed=seed) + candidate = learn_epsilon_transducer_cssr( xs, ys, input_alphabet=("0", "1"), output_alphabet=("0", "1"), - Lmax=1, + max_history=1, ) if len(list(candidate.states())) == 1: return candidate @@ -65,7 +68,7 @@ def test_recovers_memoryless_channel(): @pytest.mark.parametrize("seed", [0, 1, 2]) def test_recovers_delay_memory(seed): - xs, ys = _paired_samples(Delay(1), 10000, seed=seed) + xs, ys = _paired_samples(delay(1), 10000, seed=seed) eps = EpsilonTransducer.from_paired_sequences(xs, ys, input_alphabet=("0", "1"), output_alphabet=("0", "1")) eps.validate() assert len(list(eps.states())) == 2 @@ -83,7 +86,7 @@ def test_reconstruction_reproduces_conditional_law(): def test_rejects_mismatched_lengths(): with pytest.raises(ValueError): - transcssr("010", "01") + learn_epsilon_transducer_cssr("010", "01") def _held_out_bits_per_symbol(eps: EpsilonTransducer, xs, ys, burn: int = 20) -> float: @@ -107,12 +110,12 @@ def _held_out_bits_per_symbol(eps: EpsilonTransducer, xs, ys, burn: int = 20) -> def test_recovers_two_step_delay(): """Regression: joint suffixes grew forward and successors were never truncated, - so Delay(2) gave 5-19 states that forbade valid input-output pairs.""" - xs, ys = _paired_samples(Delay(2), 10000, seed=0) - eps = transcssr(xs, ys, input_alphabet=("0", "1"), output_alphabet=("0", "1")) + so delay(2) gave 5-19 states that forbade valid input-output pairs.""" + xs, ys = _paired_samples(delay(2), 10000, seed=0) + eps = learn_epsilon_transducer_cssr(xs, ys, input_alphabet=("0", "1"), output_alphabet=("0", "1")) eps.validate() assert len(list(eps.states())) == 4 - test_xs, test_ys = _paired_samples(Delay(2), 3000, seed=1) + test_xs, test_ys = _paired_samples(delay(2), 3000, seed=1) assert _held_out_bits_per_symbol(eps, test_xs, test_ys) == pytest.approx(0.0, abs=1e-9) @@ -124,25 +127,49 @@ def _has_markov_order_selection() -> bool: @pytest.mark.parametrize("seed", [0, 1]) def test_exact_and_bonferroni_recover_delay_memory(seed): - xs, ys = _paired_samples(Delay(1), 6000, seed=seed) + xs, ys = _paired_samples(delay(1), 6000, seed=seed) for kwargs in ({"test": "exact"}, {"correction": "bonferroni"}): - eps = transcssr(xs, ys, input_alphabet=("0", "1"), output_alphabet=("0", "1"), Lmax=2, **kwargs) + eps = learn_epsilon_transducer_cssr( + xs, ys, input_alphabet=("0", "1"), output_alphabet=("0", "1"), max_history=2, **kwargs + ) eps.validate() assert len(list(eps.states())) == 2 def test_bonferroni_keeps_memoryless_channel_single_state(): - xs, ys = _paired_samples(BinaryChannel(0.1, 0.2), 4000, seed=3) - eps = transcssr( - xs, ys, input_alphabet=("0", "1"), output_alphabet=("0", "1"), Lmax=4, alpha=0.05, correction="bonferroni" + xs, ys = _paired_samples(binary_channel(0.1, 0.2), 4000, seed=3) + eps = learn_epsilon_transducer_cssr( + xs, + ys, + input_alphabet=("0", "1"), + output_alphabet=("0", "1"), + max_history=4, + alpha=0.05, + correction="bonferroni", ) assert len(list(eps.states())) == 1 with pytest.raises(ValueError, match="unknown correction"): - transcssr(xs, ys, Lmax=1, correction="holm") + learn_epsilon_transducer_cssr(xs, ys, max_history=1, correction="holm") @pytest.mark.skipif(not _has_markov_order_selection(), reason="needs dit.inference.select_markov_order") def test_auto_lmax_delay(): - xs, ys = _paired_samples(Delay(1), 6000, seed=4) - eps = transcssr(xs, ys, input_alphabet=("0", "1"), output_alphabet=("0", "1"), Lmax="auto") + xs, ys = _paired_samples(delay(1), 6000, seed=4) + eps = learn_epsilon_transducer_cssr( + xs, ys, input_alphabet=("0", "1"), output_alphabet=("0", "1"), max_history="auto" + ) assert len(list(eps.states())) == 2 + + +@settings(max_examples=200, deadline=None) +@given( + hnp.arrays(np.int64, st.tuples(st.just(2), st.integers(2, 4)), elements=st.integers(1, 40)), +) +def test_shared_g_statistic_matches_scipy_log_likelihood(table): + """The G-test shared with process CSSR is scipy's log-likelihood statistic, Yates-corrected at dof 1.""" + from scipy import stats + + from sofic.inference.cssr.significance import g_statistic + + expected, _p, _dof, _ = stats.chi2_contingency(table, lambda_="log-likelihood") + assert g_statistic(table.astype(float)) == pytest.approx(expected, rel=1e-9, abs=1e-12) diff --git a/tests/test_examples.py b/tests/test_examples.py index 60b2f2f..30b5232 100644 --- a/tests/test_examples.py +++ b/tests/test_examples.py @@ -81,7 +81,7 @@ def test_golden_mean_shift_parry_entropy(): np.array([[1, 1], [1, 0]], dtype=float), symbol_alphabet=frozenset({0, 1}), ) - assert parry.entropy_rate() == pytest.approx(tmc.topological_entropy() / np.log(2), rel=0.05) + assert parry.entropy_rate() == pytest.approx(tmc.topological_entropy(), rel=0.05) def test_butterfly_statistical_complexity(): diff --git a/tests/test_examples_nrps.py b/tests/test_examples_nrps.py index 8f5f643..a419ac8 100644 --- a/tests/test_examples_nrps.py +++ b/tests/test_examples_nrps.py @@ -5,11 +5,6 @@ import pytest from sofic.examples import even_process, golden_mean, noisy_random_phase_slip -from sofic.examples.processes import NRPS - - -def test_nrps_alias_matches_canonical_constructor(): - assert NRPS().is_equal_process(noisy_random_phase_slip()) def test_nrps_topology(): diff --git a/tests/test_hmm_inference.py b/tests/test_hmm_inference.py index db6668c..bef622d 100644 --- a/tests/test_hmm_inference.py +++ b/tests/test_hmm_inference.py @@ -6,24 +6,24 @@ import pytest from sofic.examples import fair_coin, golden_mean -from sofic.generators.hmm_inference import ( - _emission_transition_tensors, - _forward_scaled, +from sofic.generators.matrices import emission_tensors +from sofic.generators.mealy import MealyHMM +from sofic.generators.sampling import sample +from sofic.graph import ATTR_EMISSION, ATTR_PROB +from sofic.inference.hmm import ( backward, baum_welch, forward, free_parameter_labels, log_likelihood, observed_information, - sample, score, smooth, standard_errors, two_slice_marginals, viterbi, ) -from sofic.generators.mealy import MealyHMM -from sofic.graph import ATTR_EMISSION, ATTR_PROB +from sofic.inference.hmm.filtering import _forward_scaled def test_forward_coin_initial_and_likelihood(): @@ -32,7 +32,7 @@ def test_forward_coin_initial_and_likelihood(): alpha = forward(coin, observations) assert alpha.shape == (4, 1) assert alpha[0].sum() == pytest.approx(1.0, abs=1e-9) - assert alpha[-1].sum() == pytest.approx(np.exp(log_likelihood(coin, observations)), abs=1e-9) + assert alpha[-1].sum() == pytest.approx(2.0 ** log_likelihood(coin, observations), abs=1e-9) def test_backward_coin(): @@ -92,12 +92,12 @@ def test_log_likelihood_long_sequence_stays_finite(): observations = ["0", "1"] * 1500 ll = log_likelihood(coin, observations) assert np.isfinite(ll) - assert ll == pytest.approx(-3000 * np.log(2), rel=1e-9) + assert ll == pytest.approx(-3000.0, rel=1e-9) def test_forward_scaled_rows_are_normalized(): coin = fair_coin() - alpha = forward(coin, ["0", "1", "0"], scaled=True) + alpha = forward(coin, ["0", "1", "0"], normalize=True) assert alpha.shape == (4, 1) assert np.allclose(alpha.sum(axis=1), 1.0) @@ -196,7 +196,7 @@ def test_baum_welch_accepts_single_sequence(): def test_score_matches_finite_difference_gradient(): gm = golden_mean(0.4) obs = [0, 1, 0, 0, 1, 0, 1, 0] - pi, joint = _emission_transition_tensors(gm) + pi, joint = emission_tensors(gm) idx = gm.to_mealy().reindex() a, b = idx.index("A"), idx.index("B") @@ -204,7 +204,7 @@ def loglik_entry(symbol: int, i: int, j: int, value: float) -> float: perturbed = {sym: matrix.copy() for sym, matrix in joint.items()} perturbed[symbol][i, j] = value _alpha, log_scales = _forward_scaled(pi, perturbed, list(obs)) - return float(log_scales.sum()) + return float(log_scales.sum()) * np.log(2) analytic = score(gm, obs) h = 1e-6 @@ -217,7 +217,7 @@ def loglik_entry(symbol: int, i: int, j: int, value: float) -> float: def test_observed_information_matches_numeric_hessian_scalar(): gm = golden_mean(0.5) obs = [0, 1, 0, 0, 1, 0, 1, 0] - pi, joint = _emission_transition_tensors(gm) + pi, joint = emission_tensors(gm) idx = gm.to_mealy().reindex() a, b = idx.index("A"), idx.index("B") @@ -226,7 +226,7 @@ def loglik_theta(theta: float) -> float: perturbed[0][a, a] = theta perturbed[1][a, b] = 1.0 - theta _alpha, log_scales = _forward_scaled(pi, perturbed, list(obs)) - return float(log_scales.sum()) + return float(log_scales.sum()) * np.log(2) assert free_parameter_labels(gm) == [("A", 0, "A")] theta0 = 0.5 @@ -246,7 +246,7 @@ def test_observed_information_multi_parameter_symmetric_and_matches_hessian(): labels = free_parameter_labels(hmm) assert labels == [("A", 0, "A"), ("A", 1, "A")] - pi, joint = _emission_transition_tensors(hmm) + pi, joint = emission_tensors(hmm) def loglik_free(theta: np.ndarray) -> float: perturbed = {sym: matrix.copy() for sym, matrix in joint.items()} @@ -254,7 +254,7 @@ def loglik_free(theta: np.ndarray) -> float: perturbed[1][0, 0] = theta[1] perturbed[2][0, 0] = 1.0 - theta[0] - theta[1] _alpha, log_scales = _forward_scaled(pi, perturbed, list(obs)) - return float(log_scales.sum()) + return float(log_scales.sum()) * np.log(2) base = np.array([0.2, 0.3]) h = 1e-5 @@ -333,8 +333,10 @@ def test_seeded_sample_is_reproducible_across_hash_seeds(): script = ( "import numpy as np\n" - "from sofic.examples.processes import Nemo\n" - "print(''.join(Nemo().sample(200, rng=np.random.default_rng(5))[0]))\n" + "from sofic.examples import nemo_process\n" + "from sofic.examples._construction import _relabel\n" + "nemo = _relabel(nemo_process(), symbols={0: '0', 1: '1'})\n" + "print(''.join(nemo.sample(200, rng=np.random.default_rng(5))[0]))\n" ) outputs = { subprocess.run( @@ -380,3 +382,22 @@ def test_baum_welch_restarts_reproducible_and_validated(): assert log_likelihood(first, data) == pytest.approx(log_likelihood(second, data)) with pytest.raises(ValueError): baum_welch(_symmetric_two_state(), data, n_restarts=0) + + +def test_viterbi_impossible_after_first_step_has_no_path(): + gm = golden_mean(0.5) + observations = [0, 1, 1, 0] + assert log_likelihood(gm, observations) == float("-inf") + assert viterbi(gm, observations) == [] + + +def test_baum_welch_rejects_data_with_zero_probability(): + gm = golden_mean(0.5) + with pytest.raises(ValueError, match="zero probability"): + baum_welch(gm, [[1, 1], [0, 1, 1]]) + + +def test_baum_welch_warns_when_some_sequences_are_impossible(): + gm = golden_mean(0.5) + with pytest.warns(RuntimeWarning, match="zero probability"): + baum_welch(gm, [[0, 1, 0, 0], [1, 1]], max_iter=3) diff --git a/tests/test_icdfa.py b/tests/test_icdfa.py index 87ed1b8..0c60843 100644 --- a/tests/test_icdfa.py +++ b/tests/test_icdfa.py @@ -7,22 +7,22 @@ import pytest from sofic.automata.dfa import DFA -from sofic.automata.icdfa import ( +from sofic.automata.enumeration.icdfa import ( ICDFAEnumerationError, _upper_bound_at, - count_flag_sequences, count_icdfa, count_icdfa_empty, dfa_to_icdfa_string, first_icdfa_empty_string, - flags_from_string, + icdfa_count_flag_sequences, + icdfa_flags_from_string, + icdfa_next_flags, + icdfa_string_from_flags, icdfa_string_to_dfa, iter_icdfa, iter_icdfa_empty_strings, last_icdfa_empty_string, - next_flags, next_icdfa_empty_string, - string_from_flags, validate_icdfa_empty_string, ) @@ -30,8 +30,8 @@ def test_count_flag_sequences() -> None: - assert count_flag_sequences(2, 3) == 5 - assert count_flag_sequences(3, 4) == 55 + assert icdfa_count_flag_sequences(2, 3) == 5 + assert icdfa_count_flag_sequences(3, 4) == 55 def test_count_icdfa_empty() -> None: @@ -60,7 +60,7 @@ def test_first_last_boundaries() -> None: flags = [1, 3] current = list(first_icdfa_empty_string(n=3, k=2)) - transitions = list(string_from_flags(flags, n=3, k=2)) + transitions = list(icdfa_string_from_flags(flags, n=3, k=2)) last_for_flags = list(transitions) for index in range(6): if index in flags: @@ -84,10 +84,10 @@ def test_flag_iteration_matches_count() -> None: while True: count += 1 try: - next_flags(flags, k=k) + icdfa_next_flags(flags, k=k) except StopIteration: break - assert count == count_flag_sequences(k, n) + assert count == icdfa_count_flag_sequences(k, n) def test_round_trip_codec() -> None: @@ -120,8 +120,8 @@ def test_invalid_strings() -> None: def test_flags_from_string() -> None: transitions = first_icdfa_empty_string(n=3, k=2) - assert flags_from_string(transitions, n=3) == (1, 3) - rebuilt = string_from_flags((1, 3), n=3, k=2) + assert icdfa_flags_from_string(transitions, n=3) == (1, 3) + rebuilt = icdfa_string_from_flags((1, 3), n=3, k=2) assert rebuilt == transitions diff --git a/tests/test_inference_diagnostics.py b/tests/test_inference_diagnostics.py index f87c31b..5653df0 100644 --- a/tests/test_inference_diagnostics.py +++ b/tests/test_inference_diagnostics.py @@ -6,8 +6,8 @@ import pytest from sofic.examples.epsilon_machines import even_process, golden_mean -from sofic.generators.epsilon_inference import cssr -from sofic.generators.hmm_inference import sample +from sofic.generators.sampling import sample +from sofic.inference.cssr import learn_epsilon_machine_cssr from sofic.inference.diagnostics import ( goodness_of_fit, reconstruction_sweep, @@ -24,8 +24,8 @@ def even_sample(): @pytest.mark.parametrize("statistic", ["g", "entropy_rate"]) def test_goodness_of_fit_accepts_correct_machine(even_sample, statistic): - machine = cssr(even_sample, Lmax=4, alpha=0.001) - result = goodness_of_fit(machine, even_sample, L=5, statistic=statistic, n_samples=49, rng=1) + machine = learn_epsilon_machine_cssr(even_sample, max_history=4, alpha=0.001) + result = goodness_of_fit(machine, even_sample, block_length=5, statistic=statistic, n_samples=49, rng=1) assert result.pvalue > 0.05 assert result.null.shape == (49,) assert result.forbidden_words == () @@ -33,31 +33,31 @@ def test_goodness_of_fit_accepts_correct_machine(even_sample, statistic): @pytest.mark.parametrize("statistic", ["g", "entropy_rate"]) def test_goodness_of_fit_rejects_short_lmax(even_sample, statistic): - """CSSR with Lmax=1 cannot capture the even process's parity.""" - machine = cssr(even_sample, Lmax=1, alpha=0.001) - result = goodness_of_fit(machine, even_sample, L=6, statistic=statistic, n_samples=49, rng=1) + """CSSR with max_history=1 cannot capture the even process's parity.""" + machine = learn_epsilon_machine_cssr(even_sample, max_history=1, alpha=0.001) + result = goodness_of_fit(machine, even_sample, block_length=6, statistic=statistic, n_samples=49, rng=1) assert result.pvalue <= 0.05 def test_goodness_of_fit_forbidden_word(): observations = [0, 1, 1, 0, 1, 0, 0, 1] * 20 - result = goodness_of_fit(golden_mean(0.5), observations, L=2, n_samples=9, rng=0) + result = goodness_of_fit(golden_mean(0.5), observations, block_length=2, n_samples=9, rng=0) assert result.value == float("inf") assert (1, 1) in result.forbidden_words assert result.pvalue == pytest.approx(0.1) with pytest.raises(ValueError): - goodness_of_fit(golden_mean(0.5), observations, L=2, statistic="nope") + goodness_of_fit(golden_mean(0.5), observations, block_length=2, statistic="nope") def test_topology_key_is_isomorphism_invariant(even_sample): - inferred = cssr(even_sample, Lmax=4, alpha=0.001) + inferred = learn_epsilon_machine_cssr(even_sample, max_history=4, alpha=0.001) assert topology_key(inferred) == topology_key(even_process(0.5)) assert topology_key(golden_mean(0.3)) == topology_key(golden_mean(0.7)) assert topology_key(golden_mean(0.5)) != topology_key(even_process(0.5)) def test_structure_stability_subsample(even_sample): - result = structure_stability(even_sample, n_resamples=12, rng=0, Lmax=4, alpha=0.001) + result = structure_stability(even_sample, n_resamples=12, rng=0, max_history=4, alpha=0.001) assert result.reference == topology_key(even_process(0.5)) assert result.reference_fraction >= 0.6 assert result.n_resamples == 12 @@ -73,15 +73,15 @@ def _has_stationary_bootstrap() -> bool: @pytest.mark.skipif(not _has_stationary_bootstrap(), reason="needs dit.inference.stationary_bootstrap") def test_structure_stability_block_runs(even_sample): result = structure_stability( - even_sample, n_resamples=4, rng=0, resample="block", mean_block_length=500, Lmax=3, alpha=0.001 + even_sample, n_resamples=4, rng=0, resample="block", mean_block_length=500, max_history=3, alpha=0.001 ) assert result.n_resamples == 4 with pytest.raises(ValueError): - structure_stability(even_sample, n_resamples=1, resample="nope", Lmax=2) + structure_stability(even_sample, n_resamples=1, resample="nope", max_history=2) def test_reconstruction_sweep_golden_mean(): observations, _ = sample(golden_mean(0.5), 4000, np.random.default_rng(2)) - sweep = reconstruction_sweep(observations, alphas=(0.01, 0.001), lmaxes=(1, 2, 3)) + sweep = reconstruction_sweep(observations, alphas=(0.01, 0.001), max_histories=(1, 2, 3)) target = topology_key(golden_mean(0.5)) assert all(key == target for key in sweep.values()) diff --git a/tests/test_information_anatomy.py b/tests/test_information_anatomy.py index 3d4a350..f817161 100644 --- a/tests/test_information_anatomy.py +++ b/tests/test_information_anatomy.py @@ -7,7 +7,6 @@ import pytest from sofic.examples import ( - NRPS, TENT_MAP_MISIUREWICZ_PARTITIONS, bernoulli, butterfly_process, @@ -16,6 +15,7 @@ golden_mean_forward, golden_mean_reverse, nemo_process, + noisy_random_phase_slip, tent_map_misiurewicz_a, tent_map_misiurewicz_bidirectional, tent_map_misiurewicz_forward, @@ -205,7 +205,7 @@ def _five_variable_processes(): "even": even_process(0.5).to_bidirectional(), "butterfly": butterfly_process().to_bidirectional(), "nemo": nemo_process().to_bidirectional(), - "nrps": NRPS().to_bidirectional(), + "nrps": noisy_random_phase_slip().to_bidirectional(), "tent": tent_map_misiurewicz_bidirectional(), } @@ -340,7 +340,7 @@ def test_golden_mean_ephemeral_is_pure_joint(): def test_nrps_ephemeral_is_pure_reverse_arrow_of_time(): """NRPS: forward-only ephemeral vanishes while reverse-only does not — an arrow of time.""" pytest.importorskip("dit") - bidir = NRPS().to_bidirectional() + bidir = noisy_random_phase_slip().to_bidirectional() r_mu = bidir.ephemeral_information() assert bidir.forward_only_structural_ephemeral() == pytest.approx(0.0, abs=1e-9) assert bidir.reverse_only_structural_ephemeral() == pytest.approx(r_mu, abs=1e-9) diff --git a/tests/test_information_diagram.py b/tests/test_information_diagram.py index 0057346..d60ae77 100644 --- a/tests/test_information_diagram.py +++ b/tests/test_information_diagram.py @@ -5,13 +5,13 @@ import pytest from sofic.examples import ( - NRPS, bernoulli, butterfly_process, even_process, golden_mean_forward, golden_mean_reverse, nemo_process, + noisy_random_phase_slip, ) from sofic.generators.bidirectional_epsilon_machine import BidirectionalEpsilonMachine from sofic.generators.information_diagram import ( @@ -32,7 +32,7 @@ def _processes() -> dict[str, BidirectionalEpsilonMachine]: "even": even_process(0.5).to_bidirectional(), "butterfly": butterfly_process().to_bidirectional(), "nemo": nemo_process().to_bidirectional(), - "nrps": NRPS().to_bidirectional(), + "nrps": noisy_random_phase_slip().to_bidirectional(), } diff --git a/tests/test_learning.py b/tests/test_learning.py index 7bfe988..6613153 100644 --- a/tests/test_learning.py +++ b/tests/test_learning.py @@ -1,27 +1,80 @@ -"""Tests for NL* learning.""" +"""Tests for NL* learning of canonical RFSAs and maximized prime átomata.""" +import pytest +from hypothesis import given, settings + +from sofic.automata.algorithms import equivalent +from sofic.automata.canonical.atomaton import MaximizedPrimeAtomaton +from sofic.automata.canonical.rfsa import CanonicalRFSA from sofic.automata.dfa import DFA -from sofic.automata.languages.base import AutomatonLanguage -from sofic.automata.learning import learn_maximized_prime_atomaton +from sofic.automata.learning.active import AutomatonEquivalenceOracle, LanguageMembershipOracle +from sofic.automata.learning.nlstar import learn_prime_atomaton_nlstar, learn_rfsa_from_language, learn_rfsa_nlstar +from sofic.automata.nfa import NFA +from sofic.testing.strategies import dfas + + +def _a_then_n_symbols(n: int) -> NFA: + nfa = NFA(input_alphabet=frozenset("ab"), initial_states=frozenset({0}), accepting_states=frozenset({n + 1})) + for state in range(n + 2): + nfa.graph.add_state(state) + nfa.add_transition(0, 0, "a") + nfa.add_transition(0, 0, "b") + nfa.add_transition(0, 1, "a") + for state in range(1, n + 1): + nfa.add_transition(state, state + 1, "a") + nfa.add_transition(state, state + 1, "b") + return nfa + + +def _signature(aut) -> tuple[int, int, int, int]: + return ( + len(list(aut.states())), + len(list(aut.transitions())), + len(aut.initial_states), + len(aut.accepting_states), + ) -def _teacher_dfa() -> AutomatonLanguage: +@pytest.mark.parametrize("n", [1, 2, 3]) +def test_nlstar_learns_small_canonical_rfsa_of_exponential_dfa(n): + target = _a_then_n_symbols(n) + learned = learn_rfsa_from_language(target, frozenset("ab")) + assert isinstance(learned, CanonicalRFSA) + assert equivalent(learned, target) + assert len(list(learned.states())) == n + 2 + assert _signature(learned) == _signature(CanonicalRFSA.from_language(target)) + + +def test_nlstar_with_bounded_oracle_on_explicit_membership(): dfa = DFA( - input_alphabet=frozenset({"a", "b"}), - initial_states=frozenset({"q0"}), - accepting_states=frozenset({"q1"}), + input_alphabet=frozenset({"a", "b"}), initial_states=frozenset({"q0"}), accepting_states=frozenset({"q1"}) ) dfa.graph.add_state("q0") dfa.graph.add_state("q1") dfa.add_transition("q0", "q1", "a") - dfa.add_transition("q0", "q0", "b") dfa.add_transition("q1", "q1", "a") - dfa.add_transition("q1", "q0", "b") - return AutomatonLanguage(dfa) + dfa.add_transition("q1", "q1", "b") + learned = learn_rfsa_from_language(set(dfa.iter_language(max_length=6)), frozenset("ab"), max_length=5) + for word in dfa.iter_language(max_length=5): + assert learned.recognizes(word) + +@settings(max_examples=40, deadline=None) +@given(dfas(min_states=1, max_states=4)) +def test_nlstar_matches_canonical_rfsa_construction(dfa): + alphabet = frozenset({"0", "1"}) + learned = learn_rfsa_nlstar(alphabet, LanguageMembershipOracle(dfa), AutomatonEquivalenceOracle(dfa, alphabet)) + assert equivalent(learned, dfa) + assert _signature(learned) == _signature(CanonicalRFSA.from_language(dfa)) -def test_learn_mpa(): - teacher = _teacher_dfa() - learned = learn_maximized_prime_atomaton(teacher, frozenset({"a", "b"})) - learned.validate() - assert learned.recognizes(("a",)) == (("a",) in teacher) + +@settings(max_examples=25, deadline=None) +@given(dfas(min_states=1, max_states=4)) +def test_prime_atomaton_learner_matches_construction(dfa): + alphabet = frozenset({"0", "1"}) + learned = learn_prime_atomaton_nlstar( + alphabet, LanguageMembershipOracle(dfa), AutomatonEquivalenceOracle(dfa, alphabet) + ) + assert isinstance(learned, MaximizedPrimeAtomaton) + assert equivalent(learned, dfa) + assert _signature(learned) == _signature(MaximizedPrimeAtomaton.from_language(dfa)) diff --git a/tests/test_measures.py b/tests/test_measures.py index 8fb4c43..e5e806b 100644 --- a/tests/test_measures.py +++ b/tests/test_measures.py @@ -80,16 +80,16 @@ def test_entropy_rate_moore(): assert hmm.entropy_rate() == pytest.approx(1.0, abs=1e-6) -def test_joint_block_distribution_respects_history_length_mealy(): +def test_joint_block_distribution_respects_block_length_mealy(): eps = _epsilon() - dist = eps.joint_block_distribution(history_length=2) + dist = eps.joint_block_distribution(block_length=3) assert all(len(outcome) == 3 for outcome in dist.outcomes) assert dist[("0", "1", "0")] == pytest.approx(0.125, abs=1e-12) def test_joint_block_distribution_supports_moore_hmms(): hmm = _moore() - dist = hmm.joint_block_distribution(history_length=2) + dist = hmm.joint_block_distribution(block_length=3) assert all(len(outcome) == 3 for outcome in dist.outcomes) assert dist[("0", "1", "0")] == pytest.approx(0.125, abs=1e-12) @@ -136,7 +136,7 @@ def test_golden_mean_shift_parry_entropy_rate(): np.array([[1, 1], [1, 0]], dtype=float), symbol_alphabet=frozenset({0, 1}), ) - assert parry.entropy_rate() == pytest.approx(tmc.topological_entropy() / np.log(2), rel=0.05) + assert parry.entropy_rate() == pytest.approx(tmc.topological_entropy(), rel=0.05) def test_collision_entropy_nmachine(): diff --git a/tests/test_model_selection.py b/tests/test_model_selection.py index 39d661b..e05f1be 100644 --- a/tests/test_model_selection.py +++ b/tests/test_model_selection.py @@ -63,9 +63,9 @@ def test_score_model_relationships(): scores = score_model(golden_mean(0.3), data) k, n, ll = scores.num_parameters, scores.num_observations, scores.log_likelihood assert n == 2000 - assert scores.aic == pytest.approx(2 * k - 2 * ll) - assert scores.bic == pytest.approx(k * np.log(n) - 2 * ll) - assert scores.mdl == pytest.approx(0.5 * k * np.log(n) - ll) + assert scores.aic == pytest.approx(2 * k - 2 * ll * np.log(2)) + assert scores.bic == pytest.approx(k * np.log(n) - 2 * ll * np.log(2)) + assert scores.mdl == pytest.approx(0.5 * k * np.log2(n) - ll) assert scores.aicc == pytest.approx(scores.aic + 2 * k * (k + 1) / (n - k - 1)) @@ -139,7 +139,7 @@ def test_waic_zero_variance_matches_deviance(): result = waic(matrix) assert result.p_waic == pytest.approx(0.0, abs=1e-12) assert result.lppd == pytest.approx(-3.5) - assert result.waic == pytest.approx(7.0) + assert result.waic == pytest.approx(7.0 * np.log(2)) def test_waic_positive_effective_parameters(): @@ -189,6 +189,7 @@ def test_rank_topological_epsilon_machines_prefers_two_states(): assert best.criterion_value < ranked[-1].criterion_value +@pytest.mark.filterwarnings("ignore:.*zero probability under the model:RuntimeWarning") def test_cross_validation_smoothing_keeps_forbidden_folds_finite(): """The golden mean forbids 11; a held-out '11' makes an unsmoothed fold -inf.""" rng = np.random.default_rng(6) diff --git a/tests/test_observation.py b/tests/test_observation.py index 5d1753b..b062b2c 100644 --- a/tests/test_observation.py +++ b/tests/test_observation.py @@ -1,7 +1,7 @@ """Tests for observation tables.""" from sofic.automata.languages.base import AutomatonLanguage -from sofic.automata.observation import ObservationTable +from sofic.automata.learning.observation import ObservationTable def test_defaults(): diff --git a/tests/test_posterior_diversity.py b/tests/test_posterior_diversity.py index 32cee03..be96465 100644 --- a/tests/test_posterior_diversity.py +++ b/tests/test_posterior_diversity.py @@ -5,8 +5,9 @@ import numpy as np import pytest -from sofic.examples import fair_coin -from sofic.examples.processes import Even, EvenRedundant +from sofic.examples import even_process, fair_coin +from sofic.examples._construction import _relabel +from sofic.examples.processes import even_redundant from sofic.inference.bayesian import ( InferEM, ModelComparisonEM, @@ -16,10 +17,14 @@ ) +def _even(): + return _relabel(even_process(), symbols={0: "0", 1: "1"}) + + def test_single_topology_has_zero_diversity(): pytest.importorskip("dit") data = list("1111101100") - comparison = ModelComparisonEM([Even()], data) + comparison = ModelComparisonEM([_even()], data) result = posterior_process_diversity(comparison) assert result.n_components == 1 assert result.process_diversity == pytest.approx(0.0, abs=1e-12) @@ -30,8 +35,8 @@ def test_single_topology_has_zero_diversity(): def test_duplicate_topology_has_zero_process_diversity(): pytest.importorskip("dit") data = list("1111101100") - even1 = Even() - even2 = Even() + even1 = _even() + even2 = _even() even2.name = "Even-copy" comparison = ModelComparisonEM([even1, even2], data) assert len(comparison.em_dict) == 2 @@ -43,19 +48,19 @@ def test_duplicate_topology_has_zero_process_diversity(): def test_same_process_different_topologies_have_low_process_diversity(): pytest.importorskip("dit") data = list("1111101100") - comparison = ModelComparisonEM([Even(), EvenRedundant()], data) + comparison = ModelComparisonEM([_even(), even_redundant()], data) assert len(comparison.em_dict) == 2 result = posterior_process_diversity(comparison) assert result.machine_diversity > 0.5 assert result.process_diversity < result.machine_diversity - different = posterior_process_diversity(ModelComparisonEM([Even(), fair_coin()], data)) + different = posterior_process_diversity(ModelComparisonEM([_even(), fair_coin()], data)) assert result.process_diversity < different.process_diversity def test_different_processes_have_positive_process_diversity(): pytest.importorskip("dit") data = list("1111101100") - comparison = ModelComparisonEM([Even(), fair_coin()], data) + comparison = ModelComparisonEM([_even(), fair_coin()], data) assert len(comparison.em_dict) == 2 result = posterior_process_diversity(comparison) assert result.process_diversity > 1e-6 @@ -64,7 +69,7 @@ def test_different_processes_have_positive_process_diversity(): def test_monte_carlo_is_reproducible_with_rng(): pytest.importorskip("dit") data = list("1111101100") - comparison = ModelComparisonEM([Even(), EvenRedundant()], data) + comparison = ModelComparisonEM([_even(), even_redundant()], data) rng = np.random.default_rng(0) first = posterior_process_diversity(comparison, method="monte_carlo", n_samples=32, rng=rng) rng = np.random.default_rng(0) @@ -77,7 +82,7 @@ def test_monte_carlo_is_reproducible_with_rng(): def test_word_length_override_is_respected(): pytest.importorskip("dit") data = list("1111101100") - comparison = ModelComparisonEM([Even(), EvenRedundant()], data) + comparison = ModelComparisonEM([_even(), even_redundant()], data) assert process_identification_word_length(comparison, word_length=2) == 2 result = posterior_process_diversity(comparison, word_length=2) assert result.word_length == 2 @@ -86,7 +91,7 @@ def test_word_length_override_is_respected(): def test_process_identification_word_length_conventions(): pytest.importorskip("dit") data = list("1111101100") - comparison = ModelComparisonEM([Even(), EvenRedundant()], data) + comparison = ModelComparisonEM([_even(), even_redundant()], data) n_max = max(len(p.dirichlet.nodes) for p in comparison.em_dict.values()) assert process_identification_word_length(comparison, convention="paz") == 2 * n_max - 1 assert process_identification_word_length(comparison, convention="conservative") == 2 * n_max + 1 @@ -97,7 +102,7 @@ def test_process_identification_word_length_conventions(): def test_model_comparison_convenience_methods_match_module(): pytest.importorskip("dit") data = list("1111101100") - comparison = ModelComparisonEM([Even(), EvenRedundant()], data) + comparison = ModelComparisonEM([_even(), even_redundant()], data) module_result = posterior_process_diversity(comparison) method_result = comparison.process_diversity() assert method_result == module_result @@ -107,7 +112,7 @@ def test_model_comparison_convenience_methods_match_module(): def test_posterior_mean_and_monte_carlo_same_order_of_magnitude(): pytest.importorskip("dit") data = list("1111101100") - comparison = ModelComparisonEM([Even(), fair_coin()], data) + comparison = ModelComparisonEM([_even(), fair_coin()], data) mean_result = posterior_process_diversity(comparison, method="posterior_mean") mc_result = posterior_process_diversity( comparison, @@ -126,7 +131,7 @@ def test_infer_em_posterior_mean_word_distribution_matches_machine(): from sofic.inference.bayesian.diversity import posterior_mean_word_distribution data = list("1111101100") - posterior = InferEM(Even(), data) + posterior = InferEM(_even(), data) length = 3 distribution = posterior_mean_word_distribution(posterior, length) start = max(posterior.start_node_probabilities(), key=posterior.start_node_probabilities().get) diff --git a/tests/test_processes_port.py b/tests/test_processes_port.py index 5c3137f..dd49c7a 100644 --- a/tests/test_processes_port.py +++ b/tests/test_processes_port.py @@ -6,44 +6,41 @@ import sofic.examples.processes as processes from sofic.automata.transducers import MealyMachine +from sofic.examples import fair_coin from sofic.generators.base import HiddenMarkovModel from sofic.graph import EPSILON def test_cmpy_process_constructor_names_are_exported(): expected = { - "ABC", - "AFC", - "AFC2", - "BandMerging", - "BeadsOnNecklace", - "BeforeAfter", - "BinaryMarkovChain", - "Butterfly", - "Cantor", - "CoupledGMPs", - "Ehrenfest", - "Even", - "FairCoin", - "GoldenMean", - "GoldenMeanGHMM", - "LogicMachine", - "Nemo", - "Odd", - "Period", - "Periodic", - "PerturbedCoin", - "RandomEven", - "RandomGoldenMean", - "RIP", - "SNS", - "UncoupledGMPs", + "afc", + "afc2", + "band_merging", + "beads_on_necklace", + "before_after", + "binary_markov_chain", + "butterfly_two_branch", + "cantor", + "coupled_gmps", + "ehrenfest", + "golden_mean_forbid_00", + "golden_mean_ghmm", + "logic_machine", + "odd", + "period", + "periodic", + "perturbed_coin", + "random_even", + "random_golden_mean", + "rip", + "sns", + "uncoupled_gmps", "uniform_mealyhmm", "uniform_mealymc", - "GMtoEven", - "BitFlip", - "BinaryChannel", - "Parity", + "gm_to_even", + "bit_flip", + "binary_channel", + "parity", } assert expected <= set(processes.__all__) for name in expected: @@ -53,27 +50,23 @@ def test_cmpy_process_constructor_names_are_exported(): @pytest.mark.parametrize( "constructor", [ - processes.ABC, - processes.BandMerging, - processes.BeadsOnNecklace, - processes.BeforeAfter, - processes.BinaryMarkovChain, - processes.Butterfly, - processes.Cantor, - processes.Ehrenfest, - processes.Even, - processes.FairCoin, - processes.GoldenMean, - processes.Nemo, - processes.Odd, - processes.PerturbedCoin, - processes.RIP, - processes.Rn1C, - processes.Rn1N, - processes.RRX, - processes.RRXRO, - processes.SNS, - processes.ThreeHundred, + processes.band_merging, + processes.beads_on_necklace, + processes.before_after, + processes.binary_markov_chain, + processes.butterfly_two_branch, + processes.cantor, + processes.ehrenfest, + processes.golden_mean_forbid_00, + processes.odd, + processes.perturbed_coin, + processes.rip, + processes.rn1c, + processes.rn1n, + processes.rrx, + processes.rrxro, + processes.sns, + processes.three_hundred, ], ) def test_default_process_constructors_validate(constructor): @@ -83,7 +76,7 @@ def test_default_process_constructors_validate(constructor): def test_golden_mean_cmpy_topology(): - gm = processes.GoldenMean(bias=0.25) + gm = processes.golden_mean_forbid_00(bias=0.25) edges = {(t.source, t.data["emission"], t.target): t.data["prob"] for t in gm.transitions()} assert edges[("A", "1", "A")] == pytest.approx(0.75) assert edges[("A", "0", "B")] == pytest.approx(0.25) @@ -110,13 +103,13 @@ def test_all_transducer_constructors_have_cmpy_style_alphabets_and_rows(): def test_delay_transducer_delays_symbols(): - delay = processes.Delay(length=2, symbols=["0", "1"]) + delay = processes.delay(length=2, symbols=["0", "1"]) assert delay.transduce(("1", "0")) == {("0", "0")} def test_cmpy_style_instance_composition_and_generator_transduction(): - transducer = processes.BitFlip().compose(processes.BitFlip()) + transducer = processes.bit_flip().compose(processes.bit_flip()) assert transducer.transduce(("0", "1")) == {("0", "1")} - output = processes.BinaryChannel(p=0.25, q=0.5).transduce_generator(processes.FairCoin()) + output = processes.binary_channel(p=0.25, q=0.5).transduce_generator(fair_coin()) assert output.word_probability(("1",)) == pytest.approx(0.375) diff --git a/tests/test_sofic_relation.py b/tests/test_product_alphabet_shift.py similarity index 59% rename from tests/test_sofic_relation.py rename to tests/test_product_alphabet_shift.py index 76b44c3..a6f5394 100644 --- a/tests/test_sofic_relation.py +++ b/tests/test_product_alphabet_shift.py @@ -1,18 +1,18 @@ -"""Tests for sofic relations (product-alphabet subshifts).""" +"""Tests for product-alphabet shifts (sofic relations).""" -from sofic import SoficRelation, SoficShift -from sofic.examples.processes import GMtoEven +from sofic import ProductAlphabetShift, SoficShift +from sofic.examples.processes import gm_to_even def test_from_transducer_symbols_are_pairs(): - rel = SoficRelation.from_transducer(GMtoEven()) + rel = ProductAlphabetShift.from_transducer(gm_to_even()) assert all(isinstance(symbol, tuple) and len(symbol) == 2 for symbol in rel.symbol_alphabet) assert rel.input_alphabet() == frozenset({"0", "1"}) assert rel.output_alphabet() == frozenset({"0", "1"}) def test_projections_are_sofic_shifts(): - rel = SoficRelation.from_transducer(GMtoEven()) + rel = ProductAlphabetShift.from_transducer(gm_to_even()) input_shift = rel.input_shift() output_shift = rel.output_shift() assert isinstance(input_shift, SoficShift) @@ -21,8 +21,8 @@ def test_projections_are_sofic_shifts(): def test_input_shift_matches_transducer_input_language(): - transducer = GMtoEven() - rel = SoficRelation.from_transducer(transducer) + transducer = gm_to_even() + rel = ProductAlphabetShift.from_transducer(transducer) input_words = set(rel.input_shift().factor_language(3)) assert input_words # non-empty # every relation input word is a valid transducer input word @@ -30,12 +30,12 @@ def test_input_shift_matches_transducer_input_language(): def test_yaml_round_trip(): - rel = SoficRelation.from_transducer(GMtoEven()) - restored = SoficRelation.from_yaml(rel.to_yaml()) - assert isinstance(restored, SoficRelation) + rel = ProductAlphabetShift.from_transducer(gm_to_even()) + restored = ProductAlphabetShift.from_yaml(rel.to_yaml()) + assert isinstance(restored, ProductAlphabetShift) assert restored.symbol_alphabet == rel.symbol_alphabet def test_transducer_bridge_method(): - rel = GMtoEven().to_sofic_relation() - assert isinstance(rel, SoficRelation) + rel = gm_to_even().to_product_alphabet_shift() + assert isinstance(rel, ProductAlphabetShift) diff --git a/tests/test_rfsa.py b/tests/test_rfsa.py index b0c3cde..dfead3d 100644 --- a/tests/test_rfsa.py +++ b/tests/test_rfsa.py @@ -1,8 +1,8 @@ """Tests for residual and canonical RFSA skeletons.""" +from sofic.automata.canonical.rfsa import CanonicalRFSA, ResidualFiniteStateAutomaton from sofic.automata.dfa import DFA -from sofic.automata.observation import ObservationTable -from sofic.automata.rfsa import CanonicalRFSA, ResidualFiniteStateAutomaton +from sofic.automata.learning.observation import ObservationTable def test_rfsa_validate(): diff --git a/tests/test_sliding_block_code.py b/tests/test_sliding_block_code.py index 3d3ac25..935ae9f 100644 --- a/tests/test_sliding_block_code.py +++ b/tests/test_sliding_block_code.py @@ -61,8 +61,32 @@ def test_to_transducer_realizes_code(): def test_memoryless_transducer_round_trip(): - from sofic.examples.processes import BitFlip + from sofic.examples.processes import bit_flip - code = BitFlip().to_sliding_block_code() + code = bit_flip().to_sliding_block_code() assert code.memory == 0 assert code.apply_word(["0"]) == ("1",) + + +def _golden_mean_shift() -> SoficShift: + shift = SoficShift(symbol_alphabet=frozenset({"0", "1"})) + shift.graph.add_state("A") + shift.graph.add_state("B") + shift.add_transition("A", "A", "0") + shift.add_transition("A", "B", "1") + shift.add_transition("B", "A", "0") + return shift + + +def test_apply_keeps_constraints_longer_than_the_window(): + identity = SlidingBlockCode({("0",): "0", ("1",): "1"}) + shift = _golden_mean_shift() + image = identity.apply(shift) + for length in range(1, 7): + assert set(image.factor_language(length)) == set(shift.factor_language(length)) + + +def test_apply_rejects_partial_block_map(): + partial = SlidingBlockCode({("0",): "0"}, input_alphabet={"0", "1"}) + with pytest.raises(ValueError, match="not in block_map"): + partial.apply(_golden_mean_shift()) diff --git a/tests/test_stack_inference.py b/tests/test_stack_inference.py index ce3d78e..4293d63 100644 --- a/tests/test_stack_inference.py +++ b/tests/test_stack_inference.py @@ -5,21 +5,21 @@ import numpy as np import pytest -from sofic.automata.papni import DyckAlphabet, is_well_matched, learn_sofic_dyck_shift_papni, papni_encode -from sofic.automata.rpni import learn_dfa_rpni +from sofic.automata.learning.papni import DyckAlphabet, encode_dyck_word, is_well_matched, learn_sofic_dyck_shift_papni +from sofic.automata.learning.rpni import learn_dfa_rpni from sofic.examples.shifts import dyck_shift_order, motzkin_shift -from sofic.generators.epsilon_inference import cssr from sofic.generators.stack_hmm import HiddenMarkovStackModel -from sofic.generators.stack_inference import ( - fit_stack_hmm_mle, - learn_stack_hmm_papni, - stack_cssr, - stack_subtree_merge, -) from sofic.inference.bayesian.stack_hmm import ( ModelComparisonStackHMM, StackHMMPosterior, ) +from sofic.inference.cssr import learn_epsilon_machine_cssr +from sofic.inference.cssr.stack import ( + learn_stack_hmm_cssr, + learn_stack_hmm_mle, + learn_stack_hmm_papni, + learn_stack_hmm_subtree, +) from sofic.shifts.dyck_enumeration import ( count_dyck_graph_strings, dyck_graph_string_to_shift, @@ -43,11 +43,11 @@ def _uniform_probabilities(shift): return {ref: 1.0 / len(refs) for ref in refs} -def test_is_well_matched_and_papni_encode(): +def test_is_well_matched_and_encode_dyck_word(): alphabet = _balanced_dyck_alphabet() assert is_well_matched(("(", ")"), alphabet) assert not is_well_matched((")", "("), alphabet) - assert papni_encode(("(", ")"), alphabet) == ("(", (")", "(")) + assert encode_dyck_word(("(", ")"), alphabet) == ("(", (")", "(")) def test_rpni_learns_balanced_parentheses_language(): @@ -72,7 +72,7 @@ def test_papni_recovers_dyck_shift_topology(): def test_fit_stack_hmm_mle_assigns_positive_mass(): shift = dyck_shift_order(1, call_symbols=("(",), return_symbols=(")",)) sequence = ("(", ")", "(", "(", ")", ")") - model = fit_stack_hmm_mle(shift, sequence) + model = learn_stack_hmm_mle(shift, sequence) model.validate() assert model.word_probability(sequence) > 0.0 @@ -101,7 +101,7 @@ def test_stack_cssr_recovers_motzkin_structure(): return_alphabet=shift.return_alphabet, internal_alphabet=shift.internal_alphabet, ) - inferred = stack_cssr(observations, alphabet=alphabet, Lmax=3, max_stack_depth=4, alpha=0.001) + inferred = learn_stack_hmm_cssr(observations, alphabet=alphabet, max_history=3, max_stack_depth=4, alpha=0.001) inferred.validate() assert inferred.matched_edges prefix = tuple(observations[:12]) @@ -119,7 +119,7 @@ def test_stack_subtree_merge_runs_on_sample(): return_alphabet=shift.return_alphabet, internal_alphabet=shift.internal_alphabet, ) - inferred = stack_subtree_merge(observations, alphabet=alphabet, L=2, max_stack_depth=3) + inferred = learn_stack_hmm_subtree(observations, alphabet=alphabet, max_history=2, max_stack_depth=3) inferred.validate() @@ -156,7 +156,7 @@ def test_dyck_graph_round_trip(): assert count_dyck_graph_strings(call_symbols=("(",), return_symbols=(")",)) > 0 -@pytest.mark.parametrize("method", ["papni", "stack_cssr", "flat_cssr"]) +@pytest.mark.parametrize("method", ["papni", "learn_stack_hmm_cssr", "flat_cssr"]) def test_benchmark_passive_paths(method: str): shift = dyck_shift_order(1, call_symbols=("(",), return_symbols=(")",)) probs = _uniform_probabilities(shift) @@ -183,10 +183,10 @@ def test_benchmark_passive_paths(method: str): if not positive: positive = [("(", ")")] inferred = learn_stack_hmm_papni(positive, alphabet=alphabet_bm, sequence=observations) - elif method == "stack_cssr": - inferred = stack_cssr(observations, alphabet=alphabet, Lmax=3, max_stack_depth=4, alpha=0.001) + elif method == "learn_stack_hmm_cssr": + inferred = learn_stack_hmm_cssr(observations, alphabet=alphabet, max_history=3, max_stack_depth=4, alpha=0.001) else: - flat = cssr(observations, Lmax=3, alpha=0.001) + flat = learn_epsilon_machine_cssr(observations, max_history=3, alpha=0.001) inferred = HiddenMarkovStackModel( call_alphabet=alphabet.call_alphabet, return_alphabet=alphabet.return_alphabet, @@ -225,9 +225,9 @@ def _held_out_bits_per_symbol(model, word) -> float: return -np.log2(model.word_probability(tuple(word))) / len(word) -@pytest.mark.parametrize("Lmax", [2, 3]) -def test_stack_cssr_matches_motzkin_likelihood(Lmax: int): - """Regression: homogenization only reached stacks of depth <= Lmax and return +@pytest.mark.parametrize("max_history", [2, 3]) +def test_stack_cssr_matches_motzkin_likelihood(max_history: int): + """Regression: homogenization only reached stacks of depth <= max_history and return edges were paired with unobserved calls, so held-out words got probability 0.""" shift = motzkin_shift() oracle = HiddenMarkovStackModel.from_sofic_dyck_shift(shift, _uniform_probabilities(shift)) @@ -238,7 +238,9 @@ def test_stack_cssr_matches_motzkin_likelihood(Lmax: int): ) observations, _ = oracle.sample(5000, rng=np.random.default_rng(0)) held_out, _ = oracle.sample(60, rng=np.random.default_rng(99)) - inferred = stack_cssr(observations, alphabet=alphabet, Lmax=Lmax, max_stack_depth=4, alpha=0.001) + inferred = learn_stack_hmm_cssr( + observations, alphabet=alphabet, max_history=max_history, max_stack_depth=4, alpha=0.001 + ) inferred.validate() assert len(list(inferred.states())) == 1 assert _held_out_bits_per_symbol(inferred, held_out) == pytest.approx( @@ -256,8 +258,35 @@ def test_stack_cssr_exact_and_bonferroni_run(): internal_alphabet=shift.internal_alphabet, ) for kwargs in ({"test": "exact"}, {"correction": "bonferroni"}): - inferred = stack_cssr(observations, alphabet=alphabet, Lmax=3, max_stack_depth=4, alpha=0.001, **kwargs) + inferred = learn_stack_hmm_cssr( + observations, alphabet=alphabet, max_history=3, max_stack_depth=4, alpha=0.001, **kwargs + ) inferred.validate() assert oracle.word_probability(tuple(observations[:12])) > 0.0 with pytest.raises(ValueError, match="unknown correction"): - stack_cssr(observations, alphabet=alphabet, Lmax=2, correction="holm") + learn_stack_hmm_cssr(observations, alphabet=alphabet, max_history=2, correction="holm") + + +@pytest.mark.parametrize("seed", [1, 5]) +def test_stack_cssr_keeps_histories_when_splitting(seed): + """Regression: determinization dropped histories that never emitted the splitting + symbol, leaving states whose successors were all unplaced (zero outgoing mass).""" + from sofic.shifts.sofic_dyck import transition_ref + + shift = motzkin_shift() + rng = np.random.default_rng(seed) + refs = [transition_ref(transition) for transition in shift.transitions()] + weights = rng.dirichlet(np.ones(len(refs))) + oracle = HiddenMarkovStackModel.from_sofic_dyck_shift(shift, dict(zip(refs, map(float, weights), strict=True))) + observations, _ = oracle.sample(4000, rng=rng) + alphabet = DyckAlphabet( + call_alphabet=shift.call_alphabet, + return_alphabet=shift.return_alphabet, + internal_alphabet=shift.internal_alphabet, + ) + inferred = learn_stack_hmm_cssr(observations, alphabet=alphabet, max_history=3, max_stack_depth=4, alpha=0.01) + inferred.validate() + held_out, _ = oracle.sample(400, rng=np.random.default_rng(100 + seed)) + windows = [tuple(held_out[i : i + 6]) for i in range(0, 390, 6)] + supported = sum(inferred.word_probability(window) > 0.0 for window in windows) + assert supported >= 0.9 * len(windows) diff --git a/tests/test_subsequential.py b/tests/test_subsequential.py index 70c2f96..25bb25b 100644 --- a/tests/test_subsequential.py +++ b/tests/test_subsequential.py @@ -5,7 +5,7 @@ import pytest from sofic.automata.subsequential import SubsequentialTransducer, WeightedFiniteStateTransducer -from sofic.examples.processes import BinaryChannel, BitFlip +from sofic.examples.processes import binary_channel, bit_flip from sofic.exceptions import SoficValidationError from sofic.properties import is_sequential_transducer, is_subsequential_transducer @@ -44,27 +44,27 @@ def test_subsequential_rejects_nondeterministic(): def test_wfst_probability_weight(): - w = WeightedFiniteStateTransducer.from_transducer(BinaryChannel(0.1, 0.2), semiring="probability") + w = WeightedFiniteStateTransducer.from_transducer(binary_channel(0.1, 0.2), semiring="probability") assert w.weight(["0"], ["0"]) == pytest.approx(0.9) assert w.weight(["0"], ["1"]) == pytest.approx(0.1) assert w.weight(["0", "1"], ["0", "1"]) == pytest.approx(0.9 * 0.8) def test_wfst_tropical_weight(): - w = WeightedFiniteStateTransducer.from_transducer(BinaryChannel(0.1, 0.2), semiring="tropical") + w = WeightedFiniteStateTransducer.from_transducer(binary_channel(0.1, 0.2), semiring="tropical") assert w.weight(["0"], ["0"]) == pytest.approx(-math.log(0.9)) assert math.isinf(w.weight(["0"], ["0", "0"])) def test_wfst_bad_semiring(): - w = WeightedFiniteStateTransducer.from_transducer(BitFlip(), semiring="probability") + w = WeightedFiniteStateTransducer.from_transducer(bit_flip(), semiring="probability") w.semiring = "nonsense" with pytest.raises(SoficValidationError): w.validate() def test_wfst_yaml_round_trip(): - w = WeightedFiniteStateTransducer.from_transducer(BinaryChannel(0.1, 0.2), semiring="tropical") + w = WeightedFiniteStateTransducer.from_transducer(binary_channel(0.1, 0.2), semiring="tropical") restored = WeightedFiniteStateTransducer.from_yaml(w.to_yaml()) assert restored.semiring == "tropical" assert restored.weight(["1"], ["1"]) == pytest.approx(w.weight(["1"], ["1"])) diff --git a/tests/test_symbolic_hmm.py b/tests/test_symbolic_hmm.py index a9a766c..0f38758 100644 --- a/tests/test_symbolic_hmm.py +++ b/tests/test_symbolic_hmm.py @@ -29,7 +29,6 @@ is_symbolic, probs_equal, ) -from sofic.generators.words import hmm_word_probability @pytest.mark.parametrize("partition", TENT_MAP_MISIUREWICZ_PARTITIONS) @@ -117,8 +116,8 @@ def test_fig6_hmm_matches_fig7_word_probabilities(): hmm = tent_map_misiurewicz_hmm() fwd = tent_map_misiurewicz_forward() for word in [(0, 0), (0, 1), (1, 0), (1, 1), (0, 1, 0), (1, 1, 0)]: - assert hmm_word_probability(hmm, word) == pytest.approx( - hmm_word_probability(fwd, word), + assert hmm.word_probability(word) == pytest.approx( + fwd.word_probability(word), abs=1e-10, ) diff --git a/tests/test_testing_strategies.py b/tests/test_testing_strategies.py index 0db589e..a55e429 100644 --- a/tests/test_testing_strategies.py +++ b/tests/test_testing_strategies.py @@ -5,7 +5,7 @@ import pytest from hypothesis import given, settings -from sofic.automata.icdfa import dfa_to_icdfa_string +from sofic.automata.enumeration.icdfa import dfa_to_icdfa_string from sofic.testing.strategies import dfas, epsilon_machines diff --git a/tests/test_textile.py b/tests/test_textile.py index 79de4d8..860ddf3 100644 --- a/tests/test_textile.py +++ b/tests/test_textile.py @@ -3,37 +3,37 @@ import pytest from sofic import SoficShift, TextileSystem -from sofic.examples.processes import BinaryChannel, BitFlip, SlidingNOR +from sofic.examples.processes import binary_channel, bit_flip, sliding_nor from sofic.shifts.sliding_block_code import SlidingBlockCode def test_induced_code_recovers_memory(): - textile = TextileSystem.from_transducer(SlidingNOR()) + textile = TextileSystem.from_transducer(sliding_nor()) code = textile.induced_code() assert isinstance(code, SlidingBlockCode) assert code.memory == 1 def test_induced_code_memoryless(): - textile = TextileSystem.from_transducer(BitFlip()) + textile = TextileSystem.from_transducer(bit_flip()) code = textile.induced_code() assert code.memory == 0 assert code.apply_word(["0"]) == ("1",) def test_input_output_shifts(): - textile = TextileSystem.from_transducer(SlidingNOR()) + textile = TextileSystem.from_transducer(sliding_nor()) assert isinstance(textile.input_shift(), SoficShift) assert isinstance(textile.output_shift(), SoficShift) def test_to_transducer_round_trip(): - textile = TextileSystem.from_transducer(SlidingNOR()) + textile = TextileSystem.from_transducer(sliding_nor()) machine = textile.to_transducer() - assert len(list(machine.states())) == len(list(SlidingNOR().states())) + assert len(list(machine.states())) == len(list(sliding_nor().states())) def test_stochastic_channel_has_no_induced_code(): - textile = TextileSystem.from_transducer(BinaryChannel(0.1, 0.2)) + textile = TextileSystem.from_transducer(binary_channel(0.1, 0.2)) with pytest.raises(ValueError): textile.induced_code() diff --git a/tests/test_tikz.py b/tests/test_tikz.py index 422a990..34cf7d3 100644 --- a/tests/test_tikz.py +++ b/tests/test_tikz.py @@ -61,6 +61,37 @@ def test_sofic_dyck_tikz_marks_matched_edges(): assert tikz.count("m_{1}") == 2 +def test_vpa_tikz_separates_kind_from_mid(): + from sofic.automata.vpa import VisiblyPushdownAutomaton + + vpa = VisiblyPushdownAutomaton(call_alphabet=frozenset({"a"}), stack_alphabet=frozenset({"A"})) + vpa.graph.add_state("q") + vpa.add_call_transition("q", "q", "a", "A") + + tikz = model_to_tikz(vpa) + assert r"$a\mid call\mid \uparrow A$" in tikz + assert r"\midcall" not in tikz + + +def test_tmc_tikz_multiplicity_is_math_times(): + from sofic.graph import ATTR_MULTIPLICITY, ATTR_SYMBOL + from sofic.shifts.tmc import TopologicalMarkovChain + + tmc = TopologicalMarkovChain(symbol_alphabet=frozenset({"a"})) + tmc.graph.add_state("s") + tmc.graph.add_transition("s", "s", **{ATTR_SYMBOL: "a", ATTR_MULTIPLICITY: 2}) + + assert r"$a\mid\times 2$" in model_to_tikz(tmc) + + +def test_symbolic_prob_latex_is_not_escaped(): + sp = pytest.importorskip("sympy") + from sofic.viz._tikz_format import format_prob_latex + + a = sp.Symbol("a", positive=True) + assert format_prob_latex(a / (a + 1)) == r"\frac{a}{a + 1}" + + def test_bidirectional_tikz_uses_edge_labels(): tikz = model_to_tikz(golden_mean_bidirectional(0.5), style="paper") assert r"\Edge{" in tikz diff --git a/tests/test_tmc.py b/tests/test_tmc.py index 69d3b8a..0daa712 100644 --- a/tests/test_tmc.py +++ b/tests/test_tmc.py @@ -31,4 +31,4 @@ def test_parry_measure(): tmc = TopologicalMarkovChain.from_adjacency(np.array([[0, 1], [1, 1]]), symbol_alphabet=frozenset({"0", "1"})) parry = tmc.parry_measure() parry.validate() - assert parry.entropy_rate() == pytest.approx(tmc.topological_entropy() / np.log(2), rel=0.1) + assert parry.entropy_rate() == pytest.approx(tmc.topological_entropy(), rel=0.1) diff --git a/tests/test_topological_anatomy.py b/tests/test_topological_anatomy.py index 56855da..fb4967c 100644 --- a/tests/test_topological_anatomy.py +++ b/tests/test_topological_anatomy.py @@ -7,7 +7,6 @@ import numpy as np import pytest -from sofic.exceptions import UnifilarityError from sofic.generators.epsilon_machine import EpsilonMachine from sofic.shifts.sofic import SoficShift from sofic.shifts.tmc import TopologicalMarkovChain @@ -88,7 +87,7 @@ def test_split_refines_h_top(): def test_h_top_matches_topological_entropy_and_is_additive(builder): shift = builder() anatomy = shift.topological_anatomy() - assert anatomy["h_top"] == pytest.approx(shift.topological_entropy() / np.log(2), abs=1e-9) + assert anatomy["h_top"] == pytest.approx(shift.topological_entropy(), abs=1e-9) assert anatomy["h_top"] == pytest.approx(anatomy["b_top"] + anatomy["r_top"], abs=1e-9) @@ -142,8 +141,10 @@ def test_determinizes_nondeterministic_full_shift(): assert anatomy["b_top"] == pytest.approx(0.0, abs=1e-9) -def test_non_auto_determinizable_presentation_raises(): - """A nondeterministic presentation the bounded Fischer cover cannot resolve raises.""" +def test_nondeterministic_presentation_is_resolved_by_exact_fischer_cover(): + """A presentation the old bounded follower-set cover could not resolve now works.""" + from sofic.shifts.covers import RightFischerCover + shift = SoficShift(symbol_alphabet=frozenset({0, 1})) for state in ("u", "v"): shift.graph.add_state(state) @@ -152,5 +153,8 @@ def test_non_auto_determinizable_presentation_raises(): shift.add_transition("u", "v", 1) shift.add_transition("v", "u", 0) assert not shift.is_unifilar() - with pytest.raises(UnifilarityError): - shift.topological_anatomy() + cover = RightFischerCover.from_presentation(shift) + assert cover.is_unifilar() + anatomy = shift.topological_anatomy() + assert anatomy["h_top"] == pytest.approx(cover.topological_entropy(), abs=1e-9) + assert anatomy["h_top"] == pytest.approx(anatomy["b_top"] + anatomy["r_top"], abs=1e-9) diff --git a/tests/test_topological_epsilon.py b/tests/test_topological_epsilon.py index 840f1df..1725cf6 100644 --- a/tests/test_topological_epsilon.py +++ b/tests/test_topological_epsilon.py @@ -4,7 +4,7 @@ import pytest -from sofic.automata.idfa import ( +from sofic.automata.enumeration.idfa import ( MISSING_TRANSITION, count_accessible_idfa, first_idfa_string, @@ -125,6 +125,6 @@ def test_accessible_idfa_count_grows() -> None: def count_icdfa_placeholder(k: int, n: int) -> int: - from sofic.automata.icdfa import count_icdfa_empty + from sofic.automata.enumeration.icdfa import count_icdfa_empty return count_icdfa_empty(k, n) diff --git a/tests/test_transducer_composition.py b/tests/test_transducer_composition.py index 15cc1bf..db9be07 100644 --- a/tests/test_transducer_composition.py +++ b/tests/test_transducer_composition.py @@ -5,11 +5,12 @@ import pytest import sofic.examples.processes as processes -from sofic.automata.transducer_operations import compose_tg, compose_tt, transduce_generator +from sofic.automata.transducer_operations import compose_transducer_generator, compose_transducers, transduce_generator +from sofic.examples import fair_coin def test_bitflip_composed_with_bitflip_is_identity(): - composed = compose_tt((processes.BitFlip(), processes.BitFlip())) + composed = compose_transducers((processes.bit_flip(), processes.bit_flip())) composed.validate() composed.validate_stochastic() @@ -17,36 +18,36 @@ def test_bitflip_composed_with_bitflip_is_identity(): def test_serial_composition_passes_outputs_to_next_transducer(): - composed = compose_tt((processes.GMtoEven(), processes.BitFlip())) + composed = compose_transducers((processes.gm_to_even(), processes.bit_flip())) assert composed.transduce(("0", "1")) == {("0", "0")} assert composed.transduce(("1", "0", "1")) == {("1", "0", "0")} def test_transducer_completion_emits_error_symbol_for_missing_input(): - completed = processes.GMtoEven().complete(frozenset({"0", "1"})) + completed = processes.gm_to_even().complete(frozenset({"0", "1"})) completed.validate() assert completed.transduce(("0", "0")) == {("1", "?")} def test_compose_tg_keeps_joint_input_output_emissions(): - joint = compose_tg(processes.GMtoEven(), processes.GoldenMean(0.5)) + joint = compose_transducer_generator(processes.gm_to_even(), processes.golden_mean_forbid_00(0.5)) assert joint.word_probability((("0", "1"), ("1", "1"))) == pytest.approx(1 / 3) assert joint.word_probability((("0", "1"), ("0", "1"))) == pytest.approx(0.0) def test_golden_mean_through_gm_to_even_generator(): - output = transduce_generator(processes.GMtoEven(), processes.GoldenMean(0.5)) + output = transduce_generator(processes.gm_to_even(), processes.golden_mean_forbid_00(0.5)) assert output.word_probability(("1", "1")) == pytest.approx(1 / 3) assert output.word_probability(("1", "0")) == pytest.approx(0.0) def test_binary_channel_preserves_output_probabilities(): - channel = processes.BinaryChannel(p=0.25, q=0.5) - output = transduce_generator(channel, processes.FairCoin()) + channel = processes.binary_channel(p=0.25, q=0.5) + output = transduce_generator(channel, fair_coin()) assert output.word_probability(("1",)) == pytest.approx(0.375) assert output.word_probability(("0",)) == pytest.approx(0.625) diff --git a/tests/test_vpa.py b/tests/test_vpa.py index fe5bcf5..39c0150 100644 --- a/tests/test_vpa.py +++ b/tests/test_vpa.py @@ -4,9 +4,9 @@ from sofic.automata.dfa import DFA from sofic.automata.vpa import ( - CallDrivenAutomaton, CanonicalVisiblyPushdownAutomaton, DeterministicVisiblyPushdownAutomaton, + ModularVisiblyPushdownAutomaton, MultipleEntryVisiblyPushdownAutomaton, SingleEntryVisiblyPushdownAutomaton, VisiblyPushdownAutomaton, @@ -132,8 +132,8 @@ def test_deterministic_validation_rejects_conflicting_calls(): vpa.validate() -def _cda() -> CallDrivenAutomaton: - vpa = CallDrivenAutomaton( +def _cda() -> ModularVisiblyPushdownAutomaton: + vpa = ModularVisiblyPushdownAutomaton( call_alphabet=frozenset({"c", "d"}), return_alphabet=frozenset({"r"}), stack_alphabet=frozenset({"m"}), diff --git a/tests/test_vpa_constructions.py b/tests/test_vpa_constructions.py new file mode 100644 index 0000000..cf150fe --- /dev/null +++ b/tests/test_vpa_constructions.py @@ -0,0 +1,210 @@ +"""Concrete VPA constructions checked against brute-force reference semantics.""" + +from itertools import product + +import pytest +from hypothesis import HealthCheck, given, settings +from hypothesis import strategies as st + +from sofic.automata.nwa import NestedWordAutomaton +from sofic.automata.vpa import ( + CanonicalVisiblyPushdownAutomaton, + DeterministicVisiblyPushdownAutomaton, + MultipleEntryVisiblyPushdownAutomaton, + SingleEntryVisiblyPushdownAutomaton, + VisiblyPushdownAutomaton, +) +from sofic.automata.vpa.operations import to_multiple_entry, to_single_entry +from sofic.exceptions import NonWellMatchedLanguageError + +CALLS, RETURNS, INTERNALS = frozenset({"c"}), frozenset({"r"}), frozenset({"i"}) +SYMBOLS = ("c", "r", "i") +MAX_LENGTH = 6 +WORDS = [word for n in range(MAX_LENGTH + 1) for word in product(SYMBOLS, repeat=n)] + + +@st.composite +def vpas(draw, max_states: int = 3): + """Small, possibly nondeterministic VPAs over one call, return, and internal symbol.""" + n = draw(st.integers(1, max_states)) + states = list(range(n)) + with_bottom = draw(st.booleans()) + vpa = VisiblyPushdownAutomaton( + call_alphabet=CALLS, + return_alphabet=RETURNS, + internal_alphabet=INTERNALS, + stack_alphabet=frozenset({"A", "B"} | ({"Z"} if with_bottom else set())), + bottom_stack_symbol="Z" if with_bottom else None, + initial_state=0, + accepting_states=frozenset(draw(st.sets(st.sampled_from(states)))), + ) + for state in states: + vpa.graph.add_state(state) + state = st.sampled_from(states) + for source, target in draw(st.lists(st.tuples(state, state), max_size=4)): + vpa.add_internal_transition(source, target, "i") + for source, target, push in draw(st.lists(st.tuples(state, state, st.sampled_from("AB")), max_size=3)): + vpa.add_call_transition(source, target, "c", push) + guards = ["A", "B", None] + (["Z"] if with_bottom else []) + for source, target, guard in draw(st.lists(st.tuples(state, state, st.sampled_from(guards)), max_size=3)): + vpa.add_return_transition(source, target, "r", guard) + return vpa + + +def _language(recognize) -> frozenset[tuple[str, ...]]: + return frozenset(word for word in WORDS if recognize(word)) + + +def _concat_reference(left, right): + return lambda w: any(left(w[:k]) and right(w[k:]) for k in range(len(w) + 1)) + + +def _star_reference(inner): + def recognize(word): + accepted = [True] + [False] * len(word) + for end in range(1, len(word) + 1): + accepted[end] = any(accepted[start] and inner(word[start:end]) for start in range(end)) + return accepted[-1] + + return recognize + + +SETTINGS = settings(max_examples=40, deadline=None, suppress_health_check=[HealthCheck.too_slow]) + + +@SETTINGS +@given(vpas(), vpas()) +def test_boolean_operations_match_reference(left, right): + a, b = _language(left.recognizes), _language(right.recognizes) + assert _language(left.union(right).recognizes) == a | b + assert _language(left.intersection(right).recognizes) == a & b + assert _language(left.difference(right).recognizes) == a - b + assert _language(left.complement().recognizes) == frozenset(WORDS) - a + + +@SETTINGS +@given(vpas(), vpas()) +def test_concatenation_matches_reference(left, right): + reference = _concat_reference(left.recognizes, right.recognizes) + assert _language(left.concat(right).recognizes) == _language(reference) + + +@SETTINGS +@given(vpas(max_states=2)) +def test_kleene_star_matches_reference(vpa): + assert _language(vpa.kleene_star().recognizes) == _language(_star_reference(vpa.recognizes)) + + +@SETTINGS +@given(vpas()) +def test_determinization_preserves_language_and_is_deterministic(vpa): + deterministic = vpa.determinize() + assert isinstance(deterministic, DeterministicVisiblyPushdownAutomaton) + deterministic.validate() + assert _language(deterministic.recognizes) == _language(vpa.recognizes) + assert DeterministicVisiblyPushdownAutomaton.from_vpa(vpa).equivalent(vpa) + + +@SETTINGS +@given(vpas()) +def test_double_complement_and_decision_procedures(vpa): + assert vpa.complement().complement().equivalent(vpa) + assert vpa.equivalent(vpa) + assert vpa.includes(vpa.intersection(vpa.complement())) + sample_accepts = any(vpa.recognizes(word) for word in WORDS) + witness = vpa.accepted_word() + assert vpa.is_empty() == (witness is None) + if witness is not None: + assert vpa.recognizes(witness) + if sample_accepts: + assert not vpa.is_empty() + assert vpa.union(vpa.complement()).is_universal() + + +@SETTINGS +@given(vpas()) +def test_has_unmatched_word_matches_reference(vpa): + def unmatched(word): + depth = 0 + for symbol in word: + if symbol == "c": + depth += 1 + elif symbol == "r": + if depth == 0: + return True + depth -= 1 + return depth > 0 + + if any(vpa.recognizes(w) and unmatched(w) for w in WORDS): + assert vpa.has_unmatched_word() + if not vpa.has_unmatched_word(): + assert not any(vpa.recognizes(w) and unmatched(w) for w in WORDS) + + +@SETTINGS +@given(vpas()) +def test_modular_conversions_preserve_well_matched_languages(vpa): + if vpa.has_unmatched_word(): + with pytest.raises(NonWellMatchedLanguageError): + to_single_entry(vpa) + return + reference = _language(vpa.recognizes) + single = to_single_entry(vpa) + multiple = to_multiple_entry(vpa) + assert _language(single.recognizes) == reference + assert _language(multiple.recognizes) == reference + assert _language(SingleEntryVisiblyPushdownAutomaton.minimize(vpa).recognizes) == reference + assert _language(MultipleEntryVisiblyPushdownAutomaton.minimize(vpa).recognizes) == reference + canonical = CanonicalVisiblyPushdownAutomaton.from_vpa(vpa) + assert _language(canonical.recognizes) == reference + + +def _pending_call_vpa() -> VisiblyPushdownAutomaton: + vpa = VisiblyPushdownAutomaton( + call_alphabet=CALLS, + stack_alphabet=frozenset({"S"}), + initial_state=0, + accepting_states=frozenset({1}), + ) + vpa.graph.add_state(0) + vpa.graph.add_state(1) + vpa.add_call_transition(0, 1, "c", "S") + return vpa + + +def test_canonical_vpa_rejects_languages_with_pending_calls(): + vpa = _pending_call_vpa() + assert vpa.recognizes(("c",)) + with pytest.raises(NonWellMatchedLanguageError): + CanonicalVisiblyPushdownAutomaton.from_vpa(vpa) + + +def test_nondeterministic_from_vpa_determinizes(): + vpa = VisiblyPushdownAutomaton(internal_alphabet=INTERNALS, initial_state=0, accepting_states=frozenset({1})) + vpa.graph.add_state(0) + vpa.graph.add_state(1) + vpa.add_internal_transition(0, 0, "i") + vpa.add_internal_transition(0, 1, "i") + deterministic = DeterministicVisiblyPushdownAutomaton.from_vpa(vpa) + deterministic.validate() + assert deterministic.recognizes(("i", "i")) + assert not deterministic.recognizes(()) + + +def test_nwa_operations_delegate_to_vpas(): + nwa = NestedWordAutomaton( + call_alphabet=frozenset({"x"}), + return_alphabet=frozenset({"x"}), + internal_alphabet=frozenset(), + hier_alphabet=frozenset({"H"}), + initial_state=0, + accepting_states=frozenset({0}), + ) + nwa.graph.add_state(0) + nwa.add_call_transition(0, 0, "x", "H") + nwa.add_return_transition(0, 0, "x", "H") + complement = nwa.complement() + assert not nwa.is_empty() + assert nwa.union(complement).is_universal() + assert nwa.intersection(complement).is_empty() + assert nwa.kleene_star().equivalent(nwa) diff --git a/tests/test_wheeler.py b/tests/test_wheeler.py index 1fc22de..ecaa4db 100644 --- a/tests/test_wheeler.py +++ b/tests/test_wheeler.py @@ -15,15 +15,15 @@ WheelerError, check_wheeler_axioms, colex_width, + determinize_wheeler, is_input_consistent, is_wheeler, labeled_graph, - minimum_wdfa, + minimize_wheeler, wheeler_canonical_form, wheeler_isomorphic, wheeler_order, wheeler_state_index, - wnfa_to_wdfa, ) from sofic.automata.wheeler_index import WheelerIndex from sofic.examples.epsilon_machines import ( @@ -285,18 +285,18 @@ def test_interval_power_automaton_is_polynomially_bounded(): # -- Minimization and determinization -------------------------------------- -def test_minimum_wdfa_collapses_a_de_bruijn_presentation(): +def test_minimize_wheeler_collapses_a_de_bruijn_presentation(): dfa = sigma_star_dfa(2) assert is_wheeler(dfa) - minimal = minimum_wdfa(dfa) + minimal = minimize_wheeler(dfa) assert len(list(minimal.states())) == 3 assert is_wheeler(minimal) assert equivalent(dfa, minimal, frozenset("ab")) # Already minimal, so minimizing again is a no-op. - assert len(list(minimum_wdfa(minimal).states())) == 3 + assert len(list(minimize_wheeler(minimal).states())) == 3 -def test_minimum_wdfa_rejects_non_wheeler_input(): +def test_minimize_wheeler_rejects_non_wheeler_input(): dfa = DFA(input_alphabet=BINARY, initial_states=frozenset({"A"}), accepting_states=frozenset({"A"})) for state in ("A", "B"): dfa.graph.add_state(state) @@ -304,13 +304,13 @@ def test_minimum_wdfa_rejects_non_wheeler_input(): dfa.add_transition("A", "B", 1) dfa.add_transition("B", "A", 1) with pytest.raises(WheelerError): - minimum_wdfa(dfa) + minimize_wheeler(dfa) -def test_wnfa_to_wdfa_stays_within_the_interval_bound(): +def test_determinize_wheeler_stays_within_the_interval_bound(): nfa = substring_wnfa("abra") assert is_wheeler(nfa) - dfa = wnfa_to_wdfa(nfa) + dfa = determinize_wheeler(nfa) states, arity = len(list(nfa.states())), len(nfa.input_alphabet) assert len(list(dfa.states())) <= 2 * states - 1 - arity assert is_wheeler(dfa) diff --git a/tests/test_yaml.py b/tests/test_yaml.py index da8bc67..47b68c2 100644 --- a/tests/test_yaml.py +++ b/tests/test_yaml.py @@ -5,16 +5,15 @@ import numpy as np import pytest -from sofic.automata.atomaton import Atomaton +from sofic.automata.canonical.atomaton import Atomaton from sofic.automata.dfa import DFA from sofic.automata.nfa import NFA from sofic.automata.nwa import NestedWordAutomaton from sofic.automata.transducers import MealyMachine, MooreMachine from sofic.automata.vpa import ( - CallDrivenAutomaton, CanonicalVisiblyPushdownAutomaton, - CompositeVisiblyPushdownAutomaton, DeterministicVisiblyPushdownAutomaton, + ModularVisiblyPushdownAutomaton, MultipleEntryVisiblyPushdownAutomaton, SingleEntryVisiblyPushdownAutomaton, VisiblyPushdownAutomaton, @@ -160,7 +159,7 @@ def test_vpa_variants_round_trip(): _round_trip(_base_vpa()) _round_trip(_base_vpa(DeterministicVisiblyPushdownAutomaton)) - cda = CallDrivenAutomaton( + cda = ModularVisiblyPushdownAutomaton( call_alphabet=frozenset({"c"}), return_alphabet=frozenset({"r"}), stack_alphabet=frozenset({"m"}), @@ -221,11 +220,10 @@ def test_vpa_variants_round_trip(): _round_trip(canonical) -def test_composite_vpa_round_trip(): - union = CompositeVisiblyPushdownAutomaton(operation="union", operands=(_base_vpa(), _base_vpa())) +def test_constructed_vpa_round_trip(): + union = _base_vpa().union(_base_vpa()) restored = _round_trip(union) - assert restored.operation == "union" - assert len(restored.operands) == 2 + assert restored.equivalent(union) def test_stochastic_generators_round_trip(): @@ -342,3 +340,29 @@ def test_read_write_yaml_file(tmp_path): assert type(restored) is type(eps) assert model_to_dict(restored) == model_to_dict(eps) + + +def test_cover_and_symbolic_models_round_trip(): + from sofic.shifts.base import SymbolicModel + from sofic.shifts.covers import ( + LeftFischerCover, + LeftKriegerCover, + RightFischerCover, + RightKriegerCover, + WheelerCover, + ) + + shift = SoficShift(symbol_alphabet=frozenset({"0", "1"})) + for source, target, symbol in (("A", "A", "0"), ("A", "B", "1"), ("B", "A", "0")): + shift.graph.add_state(source) + shift.graph.add_transition(source, target, **{ATTR_SYMBOL: symbol}) + for cls in (LeftFischerCover, RightFischerCover, LeftKriegerCover, RightKriegerCover, WheelerCover): + cover = cls.from_presentation(shift) + restored = _round_trip(cover) + assert type(restored) is cls + assert sorted(map(repr, restored.states())) == sorted(map(repr, cover.states())) + + bare = SymbolicModel(symbol_alphabet=frozenset({"x"})) + bare.graph.add_state(0) + bare.graph.add_transition(0, 0, **{ATTR_SYMBOL: "x"}) + assert type(_round_trip(bare)) is SymbolicModel