{"check_arguments":{"unspecified":[],"unsupported":[]},"citations":[{"contexts":["We therefore converted the graph into pairwise similarities using positive pointwise mutual information (Church and Hanks, 1990), which measures whether two words co-occur more often than expected by chance (Methods).","We estimated pairwise similarity as the Positive Pointwise Mutual Information (PPMI; Church and Hanks, 1990), where p ij = N ij / kl N kl , p i = j p ij , and s ii = \u2212 log 2 p i ."]},{"contexts":[]},{"contexts":["Among the most common statistical measures used in this task are mutual information (Church and Hanks, 1990), likelihood ratio tests (Dunning, 1993), cost criteria (Kita et al., 1994)."]},{"contexts":[]},{"contexts":[]},{"contexts":[]},{"contexts":["The t-score also associates observed counts of n-grams with expected counts (Church and Hanks, 1990), normalized by the standard deviation.","PMI measures how often words appear together compared to pure chance (Church and Hanks, 1990)."]},{"contexts":["\u2026co-occurrence matrix between the archetypes and the fluency levels, and then we re-weighted that matrix using positive pointwise information (PPMI; Church & Hanks 1990; Bullinaria & Levy 2007): Here X is the matrix of co-occurrences between archetypes and fluency levels, P ( X ij ) is the\u2026"]},{"contexts":["From an information-theoretic perspective, this log-probability difference acts as an estimator of Pointwise Mutual Information (PMI) [5], robustly capturing the information gain provided by the evidence while avoiding numerical underflow."]},{"contexts":["Leveraging the verified labels from the previous step, we construct an object co-occurrence graph and compute pairwise pointwise mutual information (PMI) [6] scores between objects (nodes) as edge scores."]},{"contexts":["Pointwise Mutual Information (Church and Hanks, 1990) is defined as: where P ( w 1 , . . . , w n ) is the observed joint probability of the sequence and (cid:81) P ( w i ) is the expected probability under lexical independence."]},{"contexts":[]},{"contexts":["\u2026and Nakajima, 2012; Wu et al., 2016) is an algorithm for vocabulary learning that iteratively chooses the merges that maximize the overall sequence likelihood the most, which can be reformulated as using the point-wise mutual information (PMI) metric (Church and Hanks, 1990) as an objective."]},{"contexts":[]},{"contexts":["(1) Edge Weight Edge weight serves as a core metric for measuring the closeness of connections between nodes in a network, providing a weighted basis for subsequent weighted degree and modularity analysis [12]."]},{"contexts":["To identify such pairs, the T -score (Church and Hanks 1990), a common collocation extraction method, is used to detect word pairs that co-occur at the ends of lines significantly more often than expected by chance."]},{"contexts":[]},{"contexts":["Edge weights are defined by the pointwise mutual information (PMI) [31] calculated strictly within the filtered dataset."]},{"contexts":[]},{"contexts":["Pointwise Mutual Information (PMI) (Church & Hanks, 1990) quantifies the association between two words w i and w j by comparing their joint probability to the product of their marginal probabilities."]},{"contexts":[]},{"contexts":["PMI measures the strength of association between a pair of discrete outcomes x and y , e.g., the co-occurrence of nodes x and y on a random walk, by comparing the log-likelihood of the observed co-occurrence of the outcomes to the log-likelihood of the outcomes occurring independently 40 : ."]},{"contexts":["8 for seven metrics) suggest findings are unlikely to be statistical artifacts, but replication with larger samples is necessary."]},{"contexts":[]},{"contexts":[]},{"contexts":["Figure 4 provides a heatmap in which the cell values give the positive pointwise mutual information (PPMI; Church & Hanks 1990; Bullinaria & Levy 2007) between the two categories, defined as PPMI ( X , a i , a j ) = max 0, P ( X ij ) P ( X i \u2217 ) \u00b7 P ( X \u2217 j ) where X is the matrix of co-occurrences\u2026"]},{"contexts":["Mutual Information in NLP. Pointwise mutual information (PMI) has been widely used in NLP for measuring word associations and semantic similarity (Church and Hanks, 1990)."]},{"contexts":[]},{"contexts":[]},{"contexts":[]},{"contexts":[]},{"contexts":["\u2026is quantified using a standardized residual (z-score): This form is closely related to Pearson residuals in contingency tables and is consistent with classical collocation statistics that separate association from marginal frequency (e.g., mutual information and likelihood-ratio tests) [7, 8]."]},{"contexts":["Traditional well-known measures such as Mutual Information ( MI ), Pointwise Mutual Information ( PMI ), or Log-Likelihood Scores ( LLS ) are primarily oriented towards collocational strength and the associative proximity of communication units [28, 29, 30]."]},{"contexts":[]},{"contexts":["The pointwise mutual information (Church & Hanks, 1990; Fano & Hawkins, 1961) between a concept \ud835\udc50 and an example \ud835\udc65 is computed as When deciding which node to expand, the \ud835\udc5d ( \ud835\udc65 ) term can be dropped as it is a constant that appears across all PMI scores; thus, this approach reduces to expanding\u2026"]},{"contexts":["Full materials (PVs, sentence stimuli and fillers) are available in Supplementary Material 1, which details the strict multistep criteria used to select target PVs and reports their string frequency and collocational strength (Mutual Information, Church & Hanks, 1990)."]},{"contexts":["\u2026research, where direct, feature-level inspection is desired. concordancing, researchers began tracing a target word\u2019s collocates and recurring patterns across corpora and time slices, often using association measures to identify characteristic collocations (Church and Hanks, 1990; Sinclair, 1991)."]},{"contexts":[]},{"contexts":[]},{"contexts":[]},{"contexts":["For this analysis, we employ mathematical tools inspired by natural language processing \u2014 specifically, term frequency \u2013 inverse document frequency (tf-idf) (Luhn, 1957; Sparck Jones, 1972) and point-wise mutual information (PMI) (Church & Hanks, 1989; Dagan et al., 1993; Niwa & Nitta, 1994)."]},{"contexts":[]},{"contexts":["Similar to Dodge et al.\u2019s prior audits of generative AI dataset filtering [22], we computed the pointwise mutual information (PMI) [17] of a regular expression (regex) appearing in the 6.5+ subset of images in LAD."]},{"contexts":["More recently, Dobrovoljc (2020) reported the evaluation of six AMs in the identi \ufb01 cation of n-grams in written and spoken corpora of Slovenian, with Dice and MI, two measures with a long-standing tradition in lexicography (Chunk and Hanks 1990; Pecina 2010; Rychl\u00fd 2008), out-performing the others."]},{"contexts":["Corpus linguists compared a target word\u2019s prominent collocates and recurring patterns across corpora and later across time slices, often supported by association measures for identifying characteristic collocations (Church and Hanks, 1990; Sinclair, 1991).","Also, most of them only focus on the salient co-occurrences with such measures as logDice (Rychl\u00fd, 2008), MI (Church and Hanks, 1990), and LLR (Dunning, 1993), thus do not measure the dynamic of the shift of the co-occurrence profile."]},{"contexts":[]},{"contexts":["Measures such as pointwise mutual information (PMI) [15] and word association strength [6] have been extensively applied to rank candidate keyphrases."]},{"contexts":["Based on the co-occurrence count A , we compute three types of co-occurrence measurements between classes: normalized count, Pearson correlation coefficient, and tanh pointwise mutual information (PMI) [49]."]},{"contexts":["The Mutual Information (MI) score quantifies the frequency of two collocates (e.g., fire and department ) appearing together within a collocation (Church & Hanks, 1990)."]},{"contexts":["I for the full NPMI matrix).","Using normalized pointwise mutual information (NPMI) (Church and Hanks, 1990; Bouma, 2009) as a measure of association between taxonomy sublabels, we observe that the overall goal of providing emotional support is strongly associated with the narrative intent of conveying a similar experience\u2026","Using normalized pointwise mutual information (NPMI) (Church and Hanks, 1990; Bouma, 2009) as a measure of association between taxonomy sublabels, we observe that the overall goal of providing emotional support is strongly associated with the narrative intent of conveying a similar experience (NPMI: 0 ."]},{"contexts":[]},{"contexts":["To address this limitation, an equivalent measure has been proposed [26]: the mutual information statistic, defined as which quantifies the strength of association on the binary log scale."]},{"contexts":[]},{"contexts":["Pointwise Mutual Information (PMI) [10] and Log-Likelihood Ratio (LLR) [11] quantify semantic proximity based on joint appearance frequencies."]},{"contexts":[]},{"contexts":[]},{"contexts":["In this study, Pointwise Mutual Information (PMI; 36) was used as a key metric to measure the statistical association between word pairs."]},{"contexts":["\u2026their ratio, or more precisely its logarithm, defined as The quantity \u2206( x t , c ) measures the conditional, point-wise mutual information [10] between the current token x t and condition c , and it quantifies how much knowing the context c increases or decreases the likelihood of\u2026"]},{"contexts":[]},{"contexts":[]},{"contexts":["We kept only those 2-and 3-word phrases with high pointwise mutual information (PMI = 5; Church and Hanks, 2002), a ratio of the probability of observing the constituent words independently."]},{"contexts":[]},{"contexts":["Pointwise mutual information (PMI) [51, 52] is a measure of the associativity between two words, calculated from the word occurrence count."]},{"contexts":["\u2026et al. (2016), which detail that the isotropy has a \u2018\u2018purification\u2019\u2019 effect that mitigates the (rather large) approximation error in the PMI models (Church and Hanks, 1990), and under-scores the power of high-dimensional geometry to retain structure through isotropic regularization in embeddings."]},{"contexts":["Based on these fundamental statistics, we derive and apply three main filtering steps to systematically determine relevant category links: 1) Positive Pointwise Mutual Information (PMI) Filtering: We compute the PMI [13] for each category pair and retain only those with a PMI greater than zero."]},{"contexts":["This measures the proximity between a new word with an unknown oil price forecasting sentiment polarity and a seed word with a known oil price forecasting sentiment polarity, thus determining the oil price forecasting sentiment category to which the new word belongs (Church and Hanks 1989).","3."]},{"contexts":["We visualize the results of the PPMI analysis in Figure 4.","We analyze demographic targeting using positive PPMI between ads and audience attributes (Fig.","A higher PPMI value indicates a stronger association between a word and a demographic group.","To understand how messages are tailored to different demographics, we look at the positive pointwise mutual information (PPMI) (Church and Hanks, 1990) between the words in the advertisements and the demographic groups.","The PPMI is a measure of association between two events, in this case, the words in the advertisements and the demographic groups.","We used the positive point-wise information (PPMI) to detect topics that appear disproportionately in ads shown to specific demographics, potentially indicating targeted messaging."]},{"contexts":["Using statistical association measures such as Pointwise Mutual Information (PMI) [8] and Lift score [9], we highlight distinctive ingredient pairings and combinations that characterize Macedonian cuisine.","To identify ingredients that co-occur more often than expected, we applied two statistical association measures: Pointwise Mutual Information (PMI) [8] and Lift [9].","We examine ingredient frequency distributions, co-occurrence patterns at the pair and triplet level, and apply Pointwise Mutual Information (PMI) [8] and Lift [9] to quantify association strength beyond raw frequency."]},{"contexts":["To construct the graph, RECAP computes pointwise mutual information (PMI) (Church and Hanks, 1990) between observation-relation pairs and candidate entities.","For a given pair ( \ud835\udc5c \ud835\udc56 , \ud835\udc5f \ud835\udc57 ) , where \ud835\udc5f \ud835\udc57 \u2208 \ud835\udc37 , we select the top-\ud835\udc3e entities \ud835\udc52 \u2217 \ud835\udc58 \u2208 \ud835\udc38 \ud835\udc47 \u222a \ud835\udc38 \ud835\udc46 that achieve the highest PMI scores: This process yields a graph that captures co-occurrence patterns between observed findings and semantic knowledge."]},{"contexts":["The concept of pointwise mutual information (PMI), defined as log p ( x | y ) p ( x ) , has been widely used in NLP tasks (Church & Hanks, 1990; Levy & Goldberg, 2014) to measure word associations."]},{"contexts":["Perception-Based Mutual Information."]},{"contexts":["\u2026meaningful regulatory relationships between transcription factors (TFs) and their target genes, we computed pointwise mutual information (PMI) [61] values for each TF\u2013gene pair, conditioned on a specified assay token a : Marginal probabilities P ( g i | a ) were obtained via single-step\u2026"]},{"contexts":["Wan et al. (2020) incorporated linguistic conceptual norms into machine learning algorithms, which leverage word norms, the informal associations be-tween closely connected words, such as nurse and doctor (Church and Hanks, 1990)."]},{"contexts":[]},{"contexts":[]},{"contexts":["\u2022 Pruning: Removing an existing point from the trajectory to eliminate redundant or noisy information."]},{"contexts":["Pointwise Mutual Information (PMI) (Fano, 1961; Church and Hanks, 1990) quantifies word associations, and has long been used in modeling lexical co-occurrence and semantic similarity."]},{"contexts":["PMI between two items i and j is defined as: where \uf028 \uf029 j i , Pr is the joint probability of items i and j co-occurring, \uf028 \uf029 i Pr and \uf028 \uf029 j Pr are the individual probabilities of items i and j occurring independently [20]."]},{"contexts":["For each category, we selected representative baselines: \u2022 PMI [39]: Pointwise Mutual Information measures the statistical association between word pairs based on their co-occurrence, often used to identify strongly related terms."]},{"contexts":["Mutual information is necessary to measure the associations between words [6]."]},{"contexts":["To correct this, we use Pointwise Mutual Information (PMI) [20] to quantify how much more often two events x and y co-occur than under independence."]},{"contexts":["Its foundational application in collocation extraction and lexicography was introduced by Church and Hanks [1].","The window size was chosen based on prior studies showing an optimal balance between semantic relevance and computational efficiency [1]."]},{"contexts":["Specifically, pointwise mutual information (MI; Church and Hanks, 1990) is used to quantify the association between two entities."]},{"contexts":[]},{"contexts":["Positive Pointwise Mutual Information (PPMI) is a common tool for measuring the association between two words in computational linguistics [Church and Hanks, 1990]."]},{"contexts":[]},{"contexts":["Meanwhile, token-deletion techniques work at the token level, using metrics such as mutual information (Church & Hanks, 1989) or perplexity (Jiang et al., 2023) to prune less informative tokens."]},{"contexts":["Sets of cognitively plausible features were originally considered an alternative to distributional embeddings [6] but now serve as a gold standard for potentially emergent knowledge in generative models."]},{"contexts":[]},{"contexts":["Since both e k and E are fixed, instead of variables, we approximate this term using Point-wise Mutual Information (PMI) and call it point-wise Total Correlation:","Therefore, we approximate this term using Point-wise Mutual Information (PMI) (Church and Hanks 1990) and solve the objective below: where s is the LLM\u2019s behavior corresponding to the candidate reflection e k , e.g. , response, self-description, or answers to multiple-choice questions."]},{"contexts":[]},{"contexts":[]},{"contexts":["\u2026et al. (2016), which details that the isotropy has a \u201cpu-rification\u201d effect that mitigates the (rather large) approximation error in the PMI models (Church and Hanks, 1990), and underscores the power of high-dimensional geometry to retain structure through isotropic regularisation in embeddings."]},{"contexts":["Statistical techniques for analyzing word associations and collocations, pioneered by Church and Hanks and extended by Navigli and Ponzetto, provide powerful tools for uncovering semantic patterns in specialized corpora [17], [18].","=3) following the methods of Church and Hanks [17]."]},{"contexts":["By leveraging pointwise mutual information, which quantifies this co-occurrence relative to an independence assumption, Church and Hanks (1990) define alternative intuitions to represent term weight within surrounding terms."]},{"contexts":["The pointwise mutual information [Church and Hanks, 1990] of domain D i followed by D j is de\ufb01ned to be where P ( D i ) is the empirical frequency of domain superfamily","In a Positive Pointwise Mutual Information (PPMI) [Church and Hanks, 1990] embedding, each dimension corresponds to a word in the vocabulary; the i th component of a word vector is a function of its frequency of co-occurrence with the i th word in the vocabulary, relative to the independent overall\u2026"]},{"contexts":["\u2026(i.e., the individual concepts in low frequency pairs are often themselves low frequency), we calculate pointwise mutual information (PMI) [19] for all concept pairs, which measures the probability of concept co-occurrence normalized by the expected probability if the concepts were\u2026","\u2026pair probabilities p D ( c 1 , c 2 ) from their constituent single concept frequencies p D ( c 1 ) , p D ( c 2 ) , we measure the pointwise mutual information (PMI) [19] between concept pairs: PMI measures how much more c 1 , c 2 co-occur in D than we would have expected them to appear by chance."]},{"contexts":[]},{"contexts":["The weight between two word nodes is then determined using positive pointwise mutual information (PPMI) [15]."]},{"contexts":["In the context of word embeddings, skip-gram with negative sampling is known to approximate point-wise mutual information log p ( x | y ) p ( x ) [7], which corresponds to the density ratio that CLIP-like models encode [16]."]}],"function":"lookup_citations","inputs":{"fields":"contexts","id":"CorpusId:9558665"},"time":0.26407504081726074}