Open Hub
Open Hub

 Settings |  Report Duplicate

0
I Use This!
Very Low Activity

Commits : Listings

Analyzed about 4 hours ago. based on code collected 1 day ago.
Nov 05, 2025 — Nov 05, 2026
Commit Message Contributor Files Modified Lines Added Lines Removed Code Location Date
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
Benchmark numbers for factoring 24 bit integer by Tile BinarySearch Optimization script have been committed to testlogs/
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
(*) Updated AsFer Design Document - benchmark numbers for both single and dual cores for tile search in computational geometric factorization (*) Updated python-src/DiscreteHyperbolicFactorizationUpperbound_TileSearch_Optimized.py for dual cores (local[2])
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
------------------------------------------------------------------------------------------------------------------------------------------- 480. (FEATURE-DONE) Computational Geometric Factorization Tiling Optimization - Binary Search for Tile Segments in Spark - 23 November 2017 ------------------------------------------------------------------------------------------------------------------------------------------- (*) Binary Search has been implemented in Spark Tile Search Optimization by parallizing the set of intervals of pixelated hyperbolic arc. (*) By this each tile interval can be binary searched in parallel to find the factor point with no necessity for global k-merge sorting. (*) Separate pixelation and tile interval file creation C++ source file has been added in cpp-src/miscellaneous (*) Arrayless tile creation has been chosen and only intervals are written to a file suffixed as .tileintervals (*) Shell script which compiles and executes the tile interval creation file and spark interval binary search script has been added. (*) Invoking the shell script as: cpp-src/miscellaneous/DiscreteHyperbolicFactorizationUpperbound_TileSearch_Optimized.sh <number_to_factorize) is sufficient to print the factors (*) Following are benchmark numbers for factorizing few reasonably high bit integers. Removing K-Merge sort has significant effect on the throughput even for a single node spark cluster on dual core (local[2]): Factorization of 12093 (14-bit): real 0m33.108s Factorization of 65532 (16-bit): real 0m44.899s Factorization of 102349 (17-bit): real 1m5.490s Factorization of 934323 (20-bit): real 8m20.017s Factorization of 1343231 (21-bit): real 16m34.759s (*) Above tile binary search numbers beat bitonic k-mergesort of tiles by many orders of magnitude (*) logs containing factors of above integers have been committed to testlogs/ (*) This optimization thus effectively supersedes mergesort of tiles and is thus a better PySpark implementation of Discrete Hyperbolic Factorization.
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
---------------------------------------------------------------------------------------------------------------------------------- Computational Geometric Factorization and Parallel Tile Search Updates ---------------------------------------------------------------------------------------------------------------------------------- (*) cpp-src/miscellaneous/DiscreteHyperbolicFactorizationUpperbound_Bitonic.cpp has been revamped and unnecessary code has been removed. Function names have been changed. NUMBER_OF_TILES has been declared as a macro and has to be manually changed and compiled to nearest power of 2 greater than number to factorize. Compilation and PySpark factorization is taken care of by shell script cpp-src/miscellaneous/DiscreteHyperbolicFactorizationUpperbound_Bitonic.sh (*) New Spark python script python-src/DiscreteHyperbolicFactorizationUpperbound_TileSearch_Optimized.py has been added to repository. This is the parallel tile search optimization already mentioned in NeuronRain AsFer Design Document. Spark parallelize() distributes the tiles-coordinates array of ordered pairs as RDDs across cloud nodes and node prints the factor point in y-axis if x-axis matches the number to factorize. Spark parallelize() simulates the binary search on a local tile segment per PRAM processor. Presently binary search has not been implemented in tilesearch() function of foreach() Spark Action. This is because map/foreach functions act on a single element argument of a parallelized RDD. (*) How Spark parallelizes a sequential datastructure on cloud nodes is equivalent to binary search on an array of length 1 which is a no-op. Because of this parallize() of Spark has been assumed to be equivalent to a binary search. (*) Benchmark numbers for two integers 1011 and 1013 with and without sorting have been committed to testlogs/ and Tiling time durations have also been captured on how well mere tile search optimizes as opposed to sorting. (*) As evidenced from python-src/testlogs/DiscreteHyperbolicFactorizationUpperbound_TileSearch_Optimized.log.22November2017, tiling preprocessing is quite insignificant (<<1 sec) and tile search for 1011 and 1013 requires approximately 20 seconds while bitonic sorting duration for same numbers is ~16minutes (960 seconds) a speedup of almost 50x. (*) Benchmarks for Local Tile Search obviate the requirement for k-merge sorting of pixelated hyperbolic tiles, if parallelism is huge. (*) Following is the trade-off: - Bitonic K-merge sorting of tiles : binary search is not necessary but a tremendous performance drag and overkill, suitable for low number of PRAMs - Local tile search : binary search is necessary only per processor but requires relatively high number of PRAMs
Srinivasan Kannan
More... almost 9 years ago
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
----------------------------------------------------------------------------------------------------------------------------------------- 476. (FEATURE-DONE and THEORY) Computational Geometric Factorization Update - Benchmarks and Tiling - 20 November 2017 - related to 34 ----------------------------------------------------------------------------------------------------------------------------------------- (*) Some further changes to Python hyperbolic tiling have been made - tile endpoints have been cast from floating point to integer for create_tile() (*) C++ tiling in cpp-src/miscellaneous/DiscreteHyperbolicFactorizationUpperbound_Bitonic.cpp has been parametrized and takes as commandline argument the integer to factorize. (*) Shell script cpp-src/miscellaneous/DiscreteHyperbolicFactorizationUpperbound_Bitonic.sh has been changed to pass in the integer to factorize to the C++ tiling binary in cpp-src/ and spark-submit for PySpark executable in python-src/ from shell args $* (*) This shell script unifies C++ tiling and PySpark factorization (*) Known issues: Python tiling still differs from C++ tiling significantly. Changing the hardcoded tile arrays storage in C++ tiling to malloc()-ed heap storage causes faulty tiling. There are only optimization issues which do not affect the factorization. Factorization works well and more benchmarks were done e.g for integers 723 and 501. Numbers for these and past numbers are charted below (*) Following profiling numbers along with previous benchmarks for 8,9,10 bits (14 November 2017) capture the trend in the exponent of logN (*) Gradual increase in exponent of number of bits because of static number of parallel RAMs (cores) is not quite steep from 3-bit to 10-bit integers as one might expect. Integers of same bit numbers need almost equal duration to factorize which implies the exponent could be a function of logN and not N. Htop shows equal loading of both cores of CPU and consumption is almost 100%+100%. (*) Cloud computing is not exactly a parallel RAM but multiple cores are PRAMs having concurrent access to a shared memory. Nodes in cloud have local memory processing too. Spark's Global State Variables (Accumulators) which are reflected across all cloud nodes are software simulations of Parallel RAMs - same global state is concurrently accessed by CPUs of spark nodes. Present PySpark implementation does the sorting on accumulators. Binary search is not necessary because Maximum elements at the end of the k-merge sorting of globalmergetiles automatically have shuffled coordinates as factors in globalcoordinates accumulator. This shaves of additional O(logN). This optimization is an update to drafts in 34.1 and 34.2: - http://sourceforge.net/projects/acadpdrafts/files/DiscreteHyperbolicPolylogarithmicSieveForIntegerFactorization_PRAM_TileMergeAndSearch_And_Stirling_Upperbound_updateddraft.tex/download - http://sourceforge.net/projects/acadpdrafts/files/DiscreteHyperbolicPolylogarithmicSieveForIntegerFactorization_PRAM_TileMergeAndSearch_And_Stirling_Upperbound_updateddraft.pdf/download
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
--------------------------------------------------------------------------------------------------------------------------------- Computational Geometric Factorization Update - Spark Accumulators, JSON for Tiling etc., --------------------------------------------------------------------------------------------------------------------------------- (*) Hyperbolic tiling code in python has been changed to reflect C++ tiling in cpp-src/miscellaneous (*) globalcoordinates global variable has been made Spark Accumulator Mutable global state - globalmergedtiles is already a Spark Accumulator global mutable state (*) bitoniclock acquire/release statements have been added for global variables, but commented for benchmarking (*) New boolean flags for bitonic comparator python style variable swap, for enabling multiple threads for assign have been added (*) merge_sorted_halves() has been invoked as a separate function (*) C++ tiling has been chosen because accuracy of long double in creating tile pixels is better than similar tiling code in python (*) Benchmark numbers for factoring 511 has been added to testlogs/ and are similar to previous numbers (*) .mergedtiles and .coordinates files are loaded/dumped as JSON in python hyperbolic tiling
Srinivasan Kannan
More... almost 9 years ago
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
-------------------------------------------------------------------------------------------------------------------------------------- Computational Geometric Factorization - Tiling Update and benchmark numbers for factoring few integers -------------------------------------------------------------------------------------------------------------------------------------- (*) Existing hyperbolic tiling code depends on C++ code in cpp-src/miscellaneous (*) To remove this dependency, python script for tiling the hyperbolic arc with simple pixelation similar to bresenham algorithm has been committed to repository - this script writes two files suffixed .coordinates and .mergedtiles from rounding off the interval [N/x,N/(x+1)] in a function hyperbolic_tiling() (*) This function is invoked in DiscreteHyperbolicFactorizationUpperbound_Bitonic_Spark.py before bitonic sort (*) DiscreteHyperbolicFactorizationUpperbound_Bitonic_Spark.py is parametrized and accepts number to factorize as commandline argument: #$SPARK_2.1.0_HOME/spark-submit DiscreteHyperbolicFactorizationUpperbound_Bitonic_Spark.py <number_to_factorize> (*) Few more time shell builtin benchmark numbers for factorizing 3-bit, 4-bit, 5-bit, 6-bit , 7-bit integers have been committed in testlogs/
Srinivasan Kannan
More... almost 9 years ago
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago
Updated AsFer Design Document
Srinivasan Kannan
as K.Srinivasan
More... almost 9 years ago