Open Hub
Open Hub

 Settings |  Report Duplicate

0
I Use This!
Very Low Activity

Commits : Listings

Analyzed about 2 hours ago. based on code collected 1 day ago.
Nov 05, 2025 — Nov 05, 2026
Commit Message Contributor Files Modified Lines Added Lines Removed Code Location Date
1547. (JIRA,THEORY and FEATURE) People Analytics - Social Network Analysis - Corrections to GitHub code search REST API endpoints and attribute paradox function arguments - related to 14,365,626,711,727,753,876,1127,1128,1129,1130,1204,1263,1269,1427,1442,1452,1532,1534 and all sections on People Analytics, Intrinsic Merit Versus Fame Equilibrium, BKS Conjecture, Stability and Noise Sensitivity of Interview LTF/PTF,Peres Theorem,Unique Identification,Talent Analytics,Code search of opensource repositories,HR Analytics and Automated Recruitment,Question-Answering,Majority Voting,Condorcet Jury Theorem Variants,Margulis-Ruzzo Thresholds,Novelty Detection,Recommender Systems,AI Question-Answering, Partial Ordered Ranking, Fame-Merit equilibrium, Intrinsic Merit as Query Complexity,Software Analytics-Mythical Manmonth-CathedralAndBazaar,Birthday Paradox - 29 August 2024
Srinivasan Kannan
More... about 2 years ago
------------------------------------------------------------------------------------------------------------------------------------------- 1547. (JIRA,THEORY and FEATURE) People Analytics - Social Network Analysis - Corrections to GitHub code search REST API endpoints and attribute paradox function arguments - related to 14,365,626,711,727,753,876,1127,1128,1129,1130,1204,1263,1269,1427,1442,1452,1532,1534 and all sections on People Analytics, Intrinsic Merit Versus Fame Equilibrium, BKS Conjecture, Stability and Noise Sensitivity of Interview LTF/PTF,Peres Theorem,Unique Identification,Talent Analytics,Code search of opensource repositories,HR Analytics and Automated Recruitment,Question-Answering,Majority Voting,Condorcet Jury Theorem Variants,Margulis-Ruzzo Thresholds,Novelty Detection,Recommender Systems,AI Question-Answering, Partial Ordered Ranking, Fame-Merit equilibrium, Intrinsic Merit as Query Complexity,Software Analytics-Mythical Manmonth-CathedralAndBazaar,Birthday Paradox - 29 August 2024 ------------------------------------------------------------------------------------------------------------------------------------------- 1547.1 python-src/SocialNetworkAnalysis_PeopleAnalytics.py has been updated to correct the GitHub code search REST API endpoints by including the query parameter and setting the user: field in curl commandlines. 1547.2 Function codesearch_statistics() is invoked by setting both opensource user id and query parameters explicitly (Query 1). 1547.3 Function attributeparadox(attribute_dict={"names":100000,"genders":3,"qualitifications":3,"photos":8000000000}) has been changed to pass on the attribute names as a dictionary argument and iterate the dictionary to print the minimum number of people required for each attribute to coincide for a given number of possible values per attribute. This simulates a Unique Identification document which contains mostly the fields - photo, date of birth, address, phone, email id - and estimates the minimum number of people required for collision in these attributes with atleast 50% probability. 1547.4 Of these attributes, photo is the most unique having possible values equal to global population of 8 billion. For example following excerpts from testlogs/SocialNetworkAnalysis_PeopleAnalytics.log.29August2024 compute minimum number of people required for 2 persons in a gathering to have same photo with atleast 50% probability as 105311 (though theoretically 2 persons having same photo on earth is 0 with 100% probability) which can be interpreted as - "In a gathering of 105311 persons atleast 2 people would look atleast 50% similar in photos" - a weird probabilistic bound for face recognition: attributeparadox(): number of names = 100000 attributeparadox(): Birthday paradox variant for names - Minimum number of people required for atleast 2 people having same names (k) with atleast 50% probability = 373.33007682967144 attributeparadox(): number of genders = 3 attributeparadox(): Birthday paradox variant for genders - Minimum number of people required for atleast 2 people having same genders (k) with atleast 50% probability = 3.099734050626334 attributeparadox(): number of qualitifications = 3 attributeparadox(): Birthday paradox variant for qualitifications - Minimum number of people required for atleast 2 people having same qualitifications (k) with atleast 50% probability = 3.099734050626334 attributeparadox(): number of photos = 8000000000 attributeparadox(): Birthday paradox variant for photos - Minimum number of people required for atleast 2 people having same photos (k) with atleast 50% probability = 105311.75391055334 1547.5 This update resolves JIRA issue https://krishnairesearch.atlassian.net/browse/NEUR-50
Srinivasan Kannan
More... about 2 years ago
Section 1546 updated
K.Srinivasan
More... about 2 years ago
Section 1546 updated
K.Srinivasan
More... about 2 years ago
Section 1543 updated
K.Srinivasan
More... about 2 years ago
+1546. (THEORY and FEATURE) GIS Urban Sprawl Analytics - Approximate and Edge Current Flow Betweenness Centralities of Live Air Traffic Dynamic Graph - 26 August 2024 - related to 823,1163,1164,1174,1175,1177,1179,1194,1211,1226,1234,1290,1291,1292,1295,1296,1297,1301,1302,1304,1312,1313,1320,1323,1326,1328,1342,1343,1349,1350,1351,1362,1363,1377,1386,1387,1395,1397,1398,1412,1414,1423,1448,1449,1462,1471,1472,1476,1529,1531,1545 and all sections on GIS Urban Sprawl Analytics, Merit of Large Scale Visuals, Gridded Population Estimators (SEDAC GPW4 and GHSL convex hull estimators), DBSCAN and Neural Network Urban Extent Learners, Computational Geometry, Intrinsic Merit Rankings, Verlhulste-R Ricker Population Estimation, Community detection, Social Network Analytics, Space filling by Curves, Unequal Circle and Polynomial packing, Computational Geometry and Polygonal packing by Voronoi Tessellation, Four color theorem, Economic Merit, Computational Economics - GDP estimation from Urban sprawl demographics, electricity consumption, Poverty alleviation, Preferential Attachment, Automatic Delineation of Urban Sprawls, Cellular Automata-SLEUTH-Polya Urn-Dynamic Facegraph Urban Growth Models,Expander Graphs,Timeseries analysis,Graph Edit Distance and Earth mover distance, Face recognition,Graph Isomorphism (Exact, Approximate and Subgraph),Graph matching,Graph similarity,Image Search,Quadrilateral Mesh Deformation and Isomorphism,Maximum Common Subgraph Problem and Approximate Topological Matching,TomTom and HERE live road traffic API,Autonomous Online Shopping Delivery by Cargo drones,EventNet GEM
K.Srinivasan
More... about 2 years ago
-------------------------------------------------------------------------------------------------------------------------------- 1546. (THEORY and FEATURE) GIS Urban Sprawl Analytics - Approximate and Edge Current Flow Betweenness Centralities of Live Air Traffic Dynamic Graph - 26 August 2024 - related to 823,1163,1164,1174,1175,1177,1179,1194,1211,1226,1234,1290,1291,1292,1295,1296,1297,1301,1302,1304,1312,1313,1320,1323,1326,1328,1342,1343,1349,1350,1351,1362,1363,1377,1386,1387,1395,1397,1398,1412,1414,1423,1448,1449,1462,1471,1472,1476,1529,1531,1545 and all sections on GIS Urban Sprawl Analytics, Merit of Large Scale Visuals, Gridded Population Estimators (SEDAC GPW4 and GHSL convex hull estimators), DBSCAN and Neural Network Urban Extent Learners, Computational Geometry, Intrinsic Merit Rankings, Verlhulste-R Ricker Population Estimation, Community detection, Social Network Analytics, Space filling by Curves, Unequal Circle and Polynomial packing, Computational Geometry and Polygonal packing by Voronoi Tessellation, Four color theorem, Economic Merit, Computational Economics - GDP estimation from Urban sprawl demographics, electricity consumption, Poverty alleviation, Preferential Attachment, Automatic Delineation of Urban Sprawls, Cellular Automata-SLEUTH-Polya Urn-Dynamic Facegraph Urban Growth Models,Expander Graphs,Timeseries analysis,Graph Edit Distance and Earth mover distance, Face recognition,Graph Isomorphism (Exact, Approximate and Subgraph),Graph matching,Graph similarity,Image Search,Quadrilateral Mesh Deformation and Isomorphism,Maximum Common Subgraph Problem and Approximate Topological Matching,TomTom and HERE live road traffic API,Autonomous Online Shopping Delivery by Cargo drones,EventNet GEM -------------------------------------------------------------------------------------------------------------------------------- 1546.1 python-src/image_pattern_mining/ImageNet/GISLiveTraffic.py has been updated to also include approximate_current_flow_betweenness_centrality() and edge_current_flow_betweenness_centrality() functions for computing approximate and edge current flow centrality measures. 1546.2 Live air traffic has been recomputed for increased number of flights (20) to increase connectivity of air traffic dynamic graph. But still they are not complex enough to get a connected air traffic graph and "not connected" exceptions for all 3 current flow betweenness centrality functions are accordingly handled in testlogs/GISLiveTraffic.log2.26August2024
K.Srinivasan
More... about 2 years ago
+1545. (THEORY and FEATURE) GIS Urban Sprawl Analytics - Time Respecting Paths and Current Flow Betweenness Centrality of Live Air Traffic Dynamic Graph - 26 August 2024 - related to 823,1163,1164,1174,1175,1177,1179,1194,1211,1226,1234,1290,1291,1292,1295,1296,1297,1301,1302,1304,1312,1313,1320,1323,1326,1328,1342,1343,1349,1350,1351,1362,1363,1377,1386,1387,1395,1397,1398,1412,1414,1423,1448,1449,1462,1471,1472,1476,1529,1531 and all sections on GIS Urban Sprawl Analytics, Merit of Large Scale Visuals, Gridded Population Estimators (SEDAC GPW4 and GHSL convex hull estimators), DBSCAN and Neural Network Urban Extent Learners, Computational Geometry, Intrinsic Merit Rankings, Verlhulste-R Ricker Population Estimation, Community detection, Social Network Analytics, Space filling by Curves, Unequal Circle and Polynomial packing, Computational Geometry and Polygonal packing by Voronoi Tessellation, Four color theorem, Economic Merit, Computational Economics - GDP estimation from Urban sprawl demographics, electricity consumption, Poverty alleviation, Preferential Attachment, Automatic Delineation of Urban Sprawls, Cellular Automata-SLEUTH-Polya Urn-Dynamic Facegraph Urban Growth Models,Expander Graphs,Timeseries analysis,Graph Edit Distance and Earth mover distance, Face recognition,Graph Isomorphism (Exact, Approximate and Subgraph),Graph matching,Graph similarity,Image Search,Quadrilateral Mesh Deformation and Isomorphism,Maximum Common Subgraph Problem and Approximate Topological Matching,TomTom and HERE live road traffic API,Autonomous Online Shopping Delivery by Cargo drones,EventNet GEM
K.Srinivasan
More... about 2 years ago
-------------------------------------------------------------------------------------------------------------------------------- 1545. (THEORY and FEATURE) GIS Urban Sprawl Analytics - Time Respecting Paths and Current Flow Betweenness Centrality of Live Air Traffic Dynamic Graph - 26 August 2024 - related to 823,1163,1164,1174,1175,1177,1179,1194,1211,1226,1234,1290,1291,1292,1295,1296,1297,1301,1302,1304,1312,1313,1320,1323,1326,1328,1342,1343,1349,1350,1351,1362,1363,1377,1386,1387,1395,1397,1398,1412,1414,1423,1448,1449,1462,1471,1472,1476,1529,1531 and all sections on GIS Urban Sprawl Analytics, Merit of Large Scale Visuals, Gridded Population Estimators (SEDAC GPW4 and GHSL convex hull estimators), DBSCAN and Neural Network Urban Extent Learners, Computational Geometry, Intrinsic Merit Rankings, Verlhulste-R Ricker Population Estimation, Community detection, Social Network Analytics, Space filling by Curves, Unequal Circle and Polynomial packing, Computational Geometry and Polygonal packing by Voronoi Tessellation, Four color theorem, Economic Merit, Computational Economics - GDP estimation from Urban sprawl demographics, electricity consumption, Poverty alleviation, Preferential Attachment, Automatic Delineation of Urban Sprawls, Cellular Automata-SLEUTH-Polya Urn-Dynamic Facegraph Urban Growth Models,Expander Graphs,Timeseries analysis,Graph Edit Distance and Earth mover distance, Face recognition,Graph Isomorphism (Exact, Approximate and Subgraph),Graph matching,Graph similarity,Image Search,Quadrilateral Mesh Deformation and Isomorphism,Maximum Common Subgraph Problem and Approximate Topological Matching,TomTom and HERE live road traffic API,Autonomous Online Shopping Delivery by Cargo drones,EventNet GEM -------------------------------------------------------------------------------------------------------------------------------- 1545.1 python-src/image_pattern_mining/ImageNet/GISLiveTraffic.py has been updated to : 1545.1.1 uncomment code for time respecting paths of air traffic dynamic graph (simple paths with temporal id) and 1545.1.2 compute current flow betweenness centrality of air traffic dynamic graph cast to an undirected graph. 1545.2 Exception for Current Flow Between Centrality has been handled when graph is not connected - Centrality measures have been re-computed in Python3.12 by venv/bin/python3.12 GISLiveTraffic.py 2>&1 > testlogs/GISLiveTraffic.log.26August2024 (with exception thrown as dynamic graph snapshot "not connected" - multiple retries might be required for current flow centrality to work as only some of those temporal air traffic graph snapshots might be connected) 1545.3 Following are some of the prominent transit airport hubs based on timestamped time respecting paths and sorted temporal centrality of dynamic air traffic graph snapshot (from testlogs/GISLiveTraffic.log.26August2024): (*) Time respecting paths in air traffic dynamic graph: {('KLU', 'JMK'): [(('KLU', 'JMK', 1724654854574058601),), (('KLU', 'JMK', 1724654902149936491),)], ('HEL', 'LHR'): [(('HEL', 'LHR', 1724654856011445265),)], ('HOQ', 'LPL'): [(('HOQ', 'LPL', 1724654857377705956),)], ('MAN', 'BOJ'): [(('MAN', 'BOJ', 1724654858865767160),)] ....] (*) Temporal Betweenness Centrality of Air Traffic Dynamic Graph: [('MEL', 0.0004995004995004995), ('SIN', 0.0004995004995004995), ('MEX', 0.0001665001665001665), ('LAX', 0.0001665001665001665),...]
K.Srinivasan
More... about 2 years ago
Section 1535 updated
Srinivasan Kannan
More... about 2 years ago
1543. (THEORY and FEATURE) Nachiketas Formal LLM - SpaCy-REBEL Knowledge Graph integration - Changes to Wikipedia Question-Answering algorithm for better accuracy 8 - related to 2,249,707,829,1133,1134,1204,1263,1269,1284,1285,1405,1413,1455,1468,1469,1536,1537,1538,1539,1540,1541,1542 and all sections on Intrinsic Merit of Texts, Recursive Lambda Function Growth and Recursive Gloss Overlap textgraph algorithms, Question-Answering, Embedding in vectorspace, Machine Translation, Transformers Attention Model,Named Entity Recognition and Meaning Representation, Complexity models of Question-Answering (multiple choice,open-ended,adaptive and static) - LTF,PTF,Switching Circuits,TQBF,Markov Chains, Large Language Models, Mildly Context Sensitive Grammars-Tree Adjoining Grammars for Natural Languages,Text restoration-Vowelless text (de)compression - 23 August 2024
Srinivasan Kannan
More... about 2 years ago
-------------------------------------------------------------------------------------------------------------------------------------------- 1543. (THEORY and FEATURE) Nachiketas Formal LLM - SpaCy-REBEL Knowledge Graph integration - Changes to Wikipedia Question-Answering algorithm for better accuracy 8 - related to 2,249,707,829,1133,1134,1204,1263,1269,1284,1285,1405,1413,1455,1468,1469,1536,1537,1538,1539,1540,1541,1542 and all sections on Intrinsic Merit of Texts, Recursive Lambda Function Growth and Recursive Gloss Overlap textgraph algorithms, Question-Answering, Embedding in vectorspace, Machine Translation, Transformers Attention Model,Named Entity Recognition and Meaning Representation, Complexity models of Question-Answering (multiple choice,open-ended,adaptive and static) - LTF,PTF,Switching Circuits,TQBF,Markov Chains, Large Language Models, Mildly Context Sensitive Grammars-Tree Adjoining Grammars for Natural Languages,Text restoration-Vowelless text (de)compression - 23 August 2024 -------------------------------------------------------------------------------------------------------------------------------------------- 1543.1 python-src/NeuronRainApps/QuestionAnswering/QuestionAnswering.py has been updated to import create_SpaCy_knowledge_graph() from python-src/KnowledgeGraph.py 1543.2 Clause for new sentence type "knowledgegraph_random_walk" has been introduced to provide Spacy-REBEL KnowledgeGraph as alternative to WordNet TextGraphs. Function create_SpaCy_knowledge_graph() implemented in 1542 is invoked for an answer text from wikipedia for a user query and Knowledge Graph instantiated by REBEL-SpaCy pipeline is processed to get random walks limited by number_of_random_walks and length of the walk limited by number_of_words_per_sentence. 1543.3 Function make_sentence() is invoked with sentence_type set to knowledgegraph_random_walk and sentences are synthesized ranked by their perplexity scores. 1543.4 Function make_sentence(wordnetsynsets,sentence_type="xtag_node34_triplets",standard_sentence_PoS_dict={"ADJ":[],"PROPN":[],"NOUN":[],"AUX":[],"ADP":[],"ADV":[],"VERB":[],"DET":[],"PRON":[],"CCONJ":[],"NUM":[],"SYM":[],"X":[]},markblanks=False,edgelabels=None) has been updated to add clause for sentence_type="knowledgegraph_random_walk" and contents of the random walks are iterated and grouped into sentences of the form headspan-relation-tailspan which is quite similar to clause for XTAG Node 34 implemented in 1536. 1543.5 QuestionAnswering function is invoked twice for same query "explain four color theorem" - first for textgraph_random_walk and second for knowledgegraph_random_walk : WikipediaRLFGTransformersQuestionAnswering(question,wsheading=True,answerslice=0.01,bothvertices_intersection=False,sentence_type="textgraph_random_walk",number_of_words_per_sentence=50,std_sentence_PoS_dict={"NUM":[],"ADJ":[],"PROPN":[],"NOUN":[],"PUNCT":[],"AUX":[],"ADP":[],"ADV":[],"VERB":[],"DET":[],"PRON":[],"CCONJ":[],"SYM":[],"X":[]},number_of_cores_per_random_walk=3,number_of_random_walks=3,blanks=False) WikipediaRLFGTransformersQuestionAnswering(question,wsheading=False,answerslice=0.5,bothvertices_intersection=False,sentence_type="knowledgegraph_random_walk",number_of_words_per_sentence=50,std_sentence_PoS_dict={"NUM":[],"ADJ":[],"PROPN":[],"NOUN":[],"PUNCT":[],"AUX":[],"ADP":[],"ADV":[],"VERB":[],"DET":[],"PRON":[],"CCONJ":[],"SYM":[],"X":[]},number_of_cores_per_random_walk=3,number_of_random_walks=3,blanks=False) 1543.6 Excerpts from python-src/NeuronRainApps/QuestionAnswering/testlogs/QuestionAnswering.KnowledgeGraph.log.23August2024 depict how perplexity scores differ for WordNet TextGraph and SpaCy-REBEL KnowledgeGraph random walk sentences for same query: ..... WordNet-ConceptNet TextGraph Random Walk - Bot generated answers ranked by perplexity: OrderedDict([(10558.7453086644, 'three basic number'), (12128.34759402403, 'four basic number'), (16240.16711019476, 'one basic number'), (35102.589080855985, 'three basic sum'), (40320.73786962726, 'four basic sum'), (53990.49754574568, 'one basic sum'), (154353.06457684588, 'one theorem sum'), (159129.83691289535, 'three cardinal sum'), (160817.73329479567, 'one cardinal sum'), (177028.9963537061, 'three theorem sum'), (177662.80860632687, 'one deducible sum'), (196300.67975645757, 'basic proposition'), (203763.163259387, 'three deducible sum'), (216504.23348702042, 'four theorem sum'), (225571.93907073708, 'four cardinal sum'), (249199.7829904695, 'four deducible sum'), (757465.2313828768, 'theorem proposition'), (935372.5125619057, 'deducible proposition')]) ..... SpaCy REBEL KnowledgeGraph Random Walk - Bot generated answers ranked by perplexity: OrderedDict([(1793.6644504710268, 'computer-assisted proof subclass of proved using a computer'), (3138.944300371735, 'adjacent subclass of boundary')]) 1543.7 Theoretically, working of the chatbot could be summarized as : 1543.7.1 query a datasource (wikipedia) for answer text which is similar to search engine results 1543.7.2 Get a knowledge graph or text graph for the answer text which goes deeper into the purport of the answer 1543.7.3 Synthesize new answer text from the textgraph or knowledge graph random walks by filling a sentence template derived from inorder traversal of a formal language grammar rule tree (XTAG for example) that results in a detailed rephrasal of wikipedia search engine result answers. 1543.7.4 Earlier algorithm converts a mildly context sensitive grammar (MCSG) natural language to a textgraph or knowledge graph, fits the random walks in the textgraphs to a matching formal grammar XTAG tree rule which is also an MCSG (which is manual at present as sentence PoS template is assumed to have been derived from XTAG,CFG,PCFG,CCG tree inorder traversal) - DisplaCy PoS dependency visualizer provides a visual left-right ordering for parts of speeches that could be useful for creating sentence templates if ordering based on grammar rules are unavailable. 1543.7.5 It is also worth mentioning that lambda function tree meaning representation of texts lifts an MCSG to a Turing Machine which is more expressive than MCSG - in other words reversing the lambda function tree structure learnt from natural text by Recursive Lambda Function Growth algorithm might involve a more expressive and meaningful natural language output than what was originally input.
Srinivasan Kannan
More... about 2 years ago
Updated index.rst
K.Srinivasan
More... about 2 years ago
Section 1540 updated
K.Srinivasan
More... about 2 years ago
Section 1529 updated
K.Srinivasan
More... about 2 years ago
Section 1535 updated
K.Srinivasan
More... about 2 years ago
Updated Requirements.txt
Srinivasan Kannan
More... about 2 years ago
1542. (THEORY and FEATURE) Meaning Representation by Textgraphs - SpaCy-REBEL Knowledge Graph implementation - Changes to Wikipedia Question-Answering algorithm for better accuracy 7 - related to 2,249,707,829,1133,1134,1204,1263,1269,1284,1285,1405,1413,1455,1468,1469 and all sections on Intrinsic Merit of Texts, Recursive Lambda Function Growth and Recursive Gloss Overlap textgraph algorithms, Question-Answering, Embedding in vectorspace, Machine Translation, Transformers Attention Model,Named Entity Recognition and Meaning Representation, Complexity models of Question-Answering (multiple choice,open-ended,adaptive and static) - LTF,PTF,Switching Circuits,TQBF,Markov Chains, Mildly Context Sensitive Grammars-Tree Adjoining Grammars for Natural Languages,Text restoration-Vowelless text (de)compression - 22 August 2024
Srinivasan Kannan
More... about 2 years ago
-------------------------------------------------------------------------------------------------------------------------------------------- 1542. (THEORY and FEATURE) Meaning Representation by Textgraphs - SpaCy-REBEL Knowledge Graph implementation - Changes to Wikipedia Question-Answering algorithm for better accuracy 7 - related to 2,249,707,829,1133,1134,1204,1263,1269,1284,1285,1405,1413,1455,1468,1469 and all sections on Intrinsic Merit of Texts, Recursive Lambda Function Growth and Recursive Gloss Overlap textgraph algorithms, Question-Answering, Embedding in vectorspace, Machine Translation, Transformers Attention Model,Named Entity Recognition and Meaning Representation, Complexity models of Question-Answering (multiple choice,open-ended,adaptive and static) - LTF,PTF,Switching Circuits,TQBF,Markov Chains, Mildly Context Sensitive Grammars-Tree Adjoining Grammars for Natural Languages,Text restoration-Vowelless text (de)compression - 22 August 2024 -------------------------------------------------------------------------------------------------------------------------------------------- 1542.1 python-src/KnowledgeGraph.py has been updated to import spacy and spacy_component (has to be downloaded from https://github.com/Babelscape/rebel/blob/main/spacy_component.py) for REBEL integration with SpaCy for relation triplets extraction and infer Knowledge graph edges from them. 1542.2 New function def create_SpaCy_knowledge_graph(text) has been defined for (*) loading spacy model en_core_web_sm (*) adding a pipeline between SpaCy and REBEL and iterate through the relation triplets via the pipeline which are added as edges of the form (head_span,tail_span) labelled by the relation to obtain a knowledge graph of the natural language text. 1542.3 DOT file of the knowledge graph is written to KnowledgeGraph.dot and Lambda functions are found from edges of the knowledge graph and their labels. 1542.4 Earlier function is invoked for a text paragraph and knowledge graph is written to KnowledgeGraph.dot (full logs in testlogs/KnowledgeGraph.log2.22August2024 and testlogs/KnowledgeGraph.log.22August2024): create_SpaCy_knowledge_graph("A large language model (LLM) is a computational model capable of language generation or other natural language processing tasks. As language models, LLMs acquire these abilities by learning statistical relationships from vast amounts of text during a self-supervised and semi-supervised training process.") 1542.5 Following relation triplets and lambda functions summarize the meaning of the above paragraph on LLMs (from testlogs/KnowledgeGraph.log2.22August2024 and testlogs/KnowledgeGraph.log.22August2024): (17, 13): {'relation': 'studies', 'head_span': natural language processing, 'tail_span': language generation} (41, 31): {'relation': 'subclass of', 'head_span': self-supervised, 'tail_span': learning} (45, 31): {'relation': 'subclass of', 'head_span': semi-supervised, 'tail_span': learning} lambda functions: ['studies(natural language processing,language generation)', 'subclass of(self-supervised,learning)', 'subclass of(semi-supervised,learning)'] 1542.6 Sentence synthesis by Nachiketas Formal LLM is either simplified or straightforward due to wide head and tail spans in SpaCy-REBEL integration whereas WordNet based textgraphs have short spans of just one word. For example, a random walk on earlier knowledge graph could be (17,13)-(45,31) which is synthesized as sentence and phrase: "1) natural language processing - studies - language generation 2) semi-supervised subclass of learning." 1542.7 Function grow_lambda_function3() in RecursiveLambdaFunctionGrowth.py already supports REBEL Knowledge Graphs as alternative to WordNet-ConceptNet textgraphs for meaning representation. Nachiketas Formal LLM in python-src/NeuronRainApps/QuestionAnswering/QuestionAnswering.py might be leveraged by Spacy-REBEL knowledge graph replacing WordNet textgraphs as perplexity score of sentences synthesized would improve but with the caveat that almost all relation extraction algorithms depend on statistical pre-trained transformer models and a grammatical representation of meaning is still unexplored. In oriental literature an ancient debate between Yaska and Panini on what gives meanings to sanskrit sentences, precedes Chomsky-Norvig debate.
Srinivasan Kannan
More... about 2 years ago
Section 1541 updated
Srinivasan Kannan
More... about 2 years ago
Section 1541 updated - 2
Srinivasan Kannan
More... about 2 years ago
Section 1541 updated
Srinivasan Kannan
More... about 2 years ago
Updated Requirements.txt
Srinivasan Kannan
More... about 2 years ago
1541. (THEORY and FEATURE) NeuronRainApps - Nachiketas Formal LLM - Textgraph Transformers Question-Answering Bot - Changes to Wikipedia Question-Answering algorithm for better accuracy 6 - ranking of bot synthesized sentences by perplexity measure of meaningfulness - related to 2,195,249,707,829,1133,1134,1204,1263,1269,1284,1285,1469,1523,1536,1537,1538,1539,1540 and all sections on Intrinsic Merit of Texts, Question-Answering, Embedding in vectorspace, Machine Translation, Transformers Attention Model,Named Entity Recognition and Meaning Representation, Complexity models of Question-Answering (multiple choice,open-ended,adaptive and static),Text Restoration,Vowelless text (de)compression - LTF,PTF,Switching Circuits,TQBF,Markov Chains - 21 August 2024
Srinivasan Kannan
More... about 2 years ago
------------------------------------------------------------------------------------------------------------------------------- 1541. (THEORY and FEATURE) NeuronRainApps - Nachiketas Formal LLM - Textgraph Transformers Question-Answering Bot - Changes to Wikipedia Question-Answering algorithm for better accuracy 6 - ranking of bot synthesized sentences by perplexity measure of meaningfulness - related to 2,195,249,707,829,1133,1134,1204,1263,1269,1284,1285,1469,1523,1536,1537,1538,1539,1540 and all sections on Intrinsic Merit of Texts, Question-Answering, Embedding in vectorspace, Machine Translation, Transformers Attention Model,Named Entity Recognition and Meaning Representation, Complexity models of Question-Answering (multiple choice,open-ended,adaptive and static),Text Restoration,Vowelless text (de)compression - LTF,PTF,Switching Circuits,TQBF,Markov Chains - 21 August 2024 ------------------------------------------------------------------------------------------------------------------------------- 1541.1 python-src/NeuronRainApps/QuestionAnswering/QuestionAnswering.py has been updated to import collections for OrderedDict and Python Perplexity implementation pyplexity for scoring English sentences by their meaningfulness. Perplexity for benchmarking LLMs is defined as: Perplexity(w1,w2,w3,...,wn) = Reciprocal(Pr(w1/w2,w3,...,wn)*Pr(w2/w3,w4,...,wn)....Pr(w(n-1)/wn)) or Reciprocal product of conditional probability of each word appearing in a sentence for a prior context set of words. Lower the perplexity score, higher is the quality of the sentence. 1541.2 Perplexity model bigrams-cord19 is loaded for prior probabilities of contexts and Perplexity measure is computed for each of the synthesized sentences from NeuronRain Formal LLM. Perplexity scores of the LLM answers are computed and stored (sorted ascending by score) in an OrderedDict. Top perplexity sentences are more suggestive of the meaning conveyed by the answer. 1541.3 LLM is invoked for 2 queries by varying the std_sentence_PoS_dict sentence template slightly by rearranging the parts-of-speech (which simulates inorder traversals of two different grammar tree rules possibly from XTAG,CFG,Probabilistic CFG,CCG etc., - Inorder traversal of rule tree is assumed to exist and beyond the scope of present LLM implementation as it depends on availability of grammar implementations. Different grammatical rules can be simulated by arbitrarily shuffling the parts-of-speech tags in std_sentence_PoS_dict sentence template parameter) 1541.4 Query 1: WikipediaRLFGTransformersQuestionAnswering(question,wsheading=True,answerslice=0.01,bothvertices_intersection=False,sentence_type="textgraph_random_walk",number_of_words_per_sentence=50,std_sentence_PoS_dict={"ADJ":[],"PROPN":[],"NOUN":[],"AUX":[],"ADP":[],"ADV":[],"VERB":[],"DET":[],"PRON":[],"CCONJ":[],"NUM":[],"SYM":[],"X":[],"PUNCT":[]},number_of_cores_per_random_walk=3,number_of_random_walks=3,blanks=False) - Query: "describe how LED TV works" - Bot generated answers ranked by perplexity from python-src/NeuronRainApps/QuestionAnswering/testlogs/QuestionAnswering.Perplexity.log.21August2024 - some phrases on internal electronics are synthesized by random walks over textgraph (ranked by perplexity score): OrderedDict([(128175.8851690847, 'rectifier tube electrodes used two'), (129044.34379728387, 'thermionic tube electrodes used two'), (147845.93010900303, 'rectifier tube diode used two'), (148847.66357452908, 'thermionic tube diode used two')]) 1541.5 Query 2 (different sentence template): WikipediaRLFGTransformersQuestionAnswering(question,wsheading=True,answerslice=0.01,bothvertices_intersection=False,sentence_type="textgraph_random_walk",number_of_words_per_sentence=50,std_sentence_PoS_dict={"NUM":[],"ADJ":[],"PROPN":[],"NOUN":[],"PUNCT":[],"AUX":[],"ADP":[],"ADV":[],"VERB":[],"DET":[],"PRON":[],"CCONJ":[],"SYM":[],"X":[]},number_of_cores_per_random_walk=3,number_of_random_walks=3,blanks=False) - Query: "Explain DSLR camera" - Bot generated answers ranked by perplexity from python-src/NeuronRainApps/QuestionAnswering/testlogs/QuestionAnswering.Perplexity.log2.21August2024 - some phrases on single lense are synthesized by random walks over textgraph (ranked by perplexity score): OrderedDict([(13449.625155160336, 'one light lens taking - usually consisting'), (13766.619456144283, 'one sensitive lens taking - usually consisting'), (18440.964489313337, 'one light lens box - usually consisting'), (18875.599698869082, 'one sensitive lens box - usually consisting'), (21076.536879606596, 'one light lens equipment - usually consisting'), (21573.28991162324, 'one sensitive lens equipment - usually consisting') 1541.6 NeuronRain Formal LLM Question-Answering bot has been christened - "Nachiketas".
Srinivasan Kannan
More... about 2 years ago
Section 1540 updated
Srinivasan Kannan
More... about 2 years ago
(*) Fix for https://krishnairesearch.atlassian.net/browse/NEUR-49 (*) Additional logs for blanks in sentence synthesis
Srinivasan Kannan
More... about 2 years ago
1540. (THEORY and FEATURE) NeuronRainApps - Textgraph Transformers Question-Answering Bot - Changes to Wikipedia Question-Answering algorithm for better accuracy 5 - phrases from multiple random walks, configurable cores per random walk, blanks and persistence to file - related to 2,195,249,707,829,1133,1134,1204,1263,1269,1284,1285,1469,1523,1536,1537,1538,1539 and all sections on Intrinsic Merit of Texts, Question-Answering, Embedding in vectorspace, Machine Translation, Transformers Attention Model,Named Entity Recognition and Meaning Representation, Complexity models of Question-Answering (multiple choice,open-ended,adaptive and static),Text Restoration,Vowelless text (de)compression - LTF,PTF,Switching Circuits,TQBF,Markov Chains - 18 August 2024
Srinivasan Kannan
More... about 2 years ago
------------------------------------------------------------------------------------------------------------------------------- 1540. (THEORY and FEATURE) NeuronRainApps - Textgraph Transformers Question-Answering Bot - Changes to Wikipedia Question-Answering algorithm for better accuracy 5 - phrases from multiple random walks, configurable cores per random walk, blanks and persistence to file - related to 2,195,249,707,829,1133,1134,1204,1263,1269,1284,1285,1469,1523,1536,1537,1538,1539 and all sections on Intrinsic Merit of Texts, Question-Answering, Embedding in vectorspace, Machine Translation, Transformers Attention Model,Named Entity Recognition and Meaning Representation, Complexity models of Question-Answering (multiple choice,open-ended,adaptive and static),Text Restoration,Vowelless text (de)compression - LTF,PTF,Switching Circuits,TQBF,Markov Chains - 18 August 2024 ------------------------------------------------------------------------------------------------------------------------------- 1540.1 python-src/NeuronRainApps/QuestionAnswering/QuestionAnswering.py has been updated to import pprint. 1540.2 def WikipediaRLFGTransformersQuestionAnswering(question,questionfraction=1,maxanswers=1,keywordsearch=False,wsheading=True,answerslice=1,answerfraction=1,bothvertices_intersection=True,sentence_type="xtag_node34_triplets",number_of_random_walks=10,number_of_words_per_sentence=5,number_of_cores_per_random_walk=5,std_sentence_PoS_dict={"ADJ":[],"PROPN":[],"NOUN":[],"AUX":[],"ADP":[],"ADV":[],"VERB":[],"DET":[],"PRON":[],"CCONJ":[],"NUM":[],"SYM":[],"X":[]},blanks=False) has been updated for additional arguments - number_of_cores_per_random_walk, number_of_random_walks and blanks respectively for limiting minimum number of core vertices in a random walk, synthesis of sentences from multiple random walks and marking unfilled part-of-speeches in synthesized sentences. 1540.3 Synthesized sentences, possibly containing blanks for unfilled parts-of-speech, are persisted to file QuestionAnswering.FormalLLM.txt which can be further processed by function grow_lambda_function3(textread,predict_word=True, fillintheblankstxt="RecursiveLambdaFunctionGrowth_FillInTheBlanks.txt") in RecursiveLambdaFunctionGrowth.py for filling the blanks approximately. 1540.4 Number of random walks has been made configurable instead of 1 earlier. Multiple random walks from answertextgraph are obtained by passing on number_of_random_walks to generate_random_paths() NetworkX API. Array of random walks are iterated and only those random walks which contain atleast number_of_cores_per_random_walk vertices are selected for sentence synthesis. 1540.5 make_sentence() is invoked with additional blanks argument newly introduced. 1540.6 Function def make_sentence(wordnetsynsets,sentence_type="xtag_node34_triplets",standard_sentence_PoS_dict={"ADJ":[],"PROPN":[],"NOUN":[],"AUX":[],"ADP":[],"ADV":[],"VERB":[],"DET":[],"PRON":[],"CCONJ":[],"NUM":[],"SYM":[],"X":[]},markblanks=False) has been changed for additional markblanks argument. 1540.7 User-defined flattened XTAG inorder traversal dictionary [inorder traversal must be manually arrived at from any XTAG tree rule of user choice or from parse trees of already available grammars in NLTK e.g CFG,CCG - RecursveLambdaFunctionGrowth.py already implements function tree_adjoining_grammar_annotated() wrapper for nltk.CFG.fromstring()] standard_sentence_PoS_dict is populated with "------" for unfilled parts-of-speech elements in the dict. 1540.8 Function WikipediaRLFGTransformersQuestionAnswering(question,wsheading=False,answerslice=0.01,bothvertices_intersection=False,sentence_type="textgraph_random_walk",number_of_words_per_sentence=50,std_sentence_PoS_dict={"ADJ":[],"PROPN":[],"NOUN":[],"AUX":[],"ADP":[],"ADV":[],"VERB":[],"DET":[],"PRON":[],"CCONJ":[],"NUM":[],"SYM":[],"X":[],"PUNCT":[]},number_of_random_walks=3,blanks=False) is invoked for 3 random walks and atleast 5 core vertices per walk with blank marking set to False. 1540.9 For query "list of natural disasters", some of the meaningful sentences and phrases synthesized that capture the meaning of "disasters" spot-on are (from QuestionAnswering.FormalLLM.txt and testlogs/QuestionAnswering.log.18August2024) while others are grammatically correct but less meaningful: irremediable extreme misfortune usually ruin irremediable extreme state usually ruin
Srinivasan Kannan
More... about 2 years ago
+1539. (THEORY and FEATURE) NeuronRainApps - Textgraph Transformers Question-Answering Bot - Changes to Wikipedia Question-Answering algorithm for better accuracy 4 - sentence synthesis based on an XTAG-like sentence template - related to 2,195,249,707,829,1133,1134,1204,1263,1269,1284,1285,1469,1523,1536,1537,1538 and all sections on Intrinsic Merit of Texts, Question-Answering, Embedding in vectorspace, Machine Translation, Transformers Attention Model,Named Entity Recognition and Meaning Representation, Complexity models of Question-Answering (multiple choice,open-ended,adaptive and static),Text Restoration,Vowelless text (de)compression - LTF,PTF,Switching Circuits,TQBF,Markov Chains - 16 August 2024
Srinivasan Kannan
More... about 2 years ago