Calculating intersection of lots of sets in R -

-

i've got long list of authors , words like

author1,word1 author1,word2 author1,word3 author2,word2 author3,word1

the actual list has hundreds of authors , thousands of words. exists csv file have read dataframe , de-duplicated like

    > typeof(x)     [1] "list"     > colnames(x)     [1] "author"   "word"

the last bit of dput(head(x)) looks like

    ), class = "factor")), .names = c("author", "word"), row.names = c(na,      6l), class = "data.frame")

what i'm trying calculate how similar word lists between authors based on intersection of author's wordlists percentage of 1 authors total vocabulary. (i'm sure there proper terms i'm doing don't quite know are.)

in python or perl group words author , use nested loops compare everyone else i'm wondering how in r? have feeling "use apply" going answer- if can please explain in small words newbies me?

here's 1 way using data.table:

## 1: generate test data set.seed(1l); wordlist <- paste0('word',1:5); authorlist <- paste0('author',1:5); rs <- sample(1:5,length(authorlist),replace=t); aw <- data.table(     author=factor(rep(authorlist,rs)),     word=factor(do.call(c,lapply(rs,function(r) sort(sample(wordlist,r))))),     key='author' ); aw; ##      author  word ##  1: author1 word4 ##  2: author1 word5 ##  3: author2 word3 ##  4: author2 word4 ##  5: author3 word1 ##  6: author3 word4 ##  7: author3 word5 ##  8: author4 word1 ##  9: author4 word2 ## 10: author4 word3 ## 11: author4 word4 ## 12: author4 word5 ## 13: author5 word2 ## 14: author5 word5 
## 2: initialize intersection table unique combinations of authors ai <- aw[,setkey(setnames(nm=c('a1','a2'),as.data.table(t(combn(unique(author),2l)))))]; 
## 3: compute word intersection size each combination of authors ai[,int:=length(intersect(aw[a1,word],aw[a2,word])),key(ai)]; ##          a1      a2 int ##  1: author1 author2   1 ##  2: author1 author3   2 ##  3: author1 author4   2 ##  4: author1 author5   1 ##  5: author2 author3   1 ##  6: author2 author4   2 ##  7: author2 author5   0 ##  8: author3 author4   3 ##  9: author3 author5   1 ## 10: author4 author5   2 
## 4: compute percentages ai[,`:=`(p1=int/aw[a1,.n],p2=int/aw[a2,.n]),key(ai)]; ##          a1      a2 int        p1        p2 ##  1: author1 author2   1 0.5000000 0.5000000 ##  2: author1 author3   2 1.0000000 0.6666667 ##  3: author1 author4   2 1.0000000 0.4000000 ##  4: author1 author5   1 0.5000000 0.5000000 ##  5: author2 author3   1 0.5000000 0.3333333 ##  6: author2 author4   2 1.0000000 0.4000000 ##  7: author2 author5   0 0.0000000 0.0000000 ##  8: author3 author4   3 1.0000000 0.6000000 ##  9: author3 author5   1 0.3333333 0.5000000 ## 10: author4 author5   2 0.4000000 1.0000000

Comments

Post a Comment

Popular posts from this blog

ios - Memory not freeing up after popping viewcontroller using ARC -

-
hi project arc based, using uinavigationcontroller make transition between viewcontroller. using profiler analyse happening behind scene memory. noticed when push viewcontroller allocate memory components , when pop not freeing allocated memory. since using arc unable implement dealloc or release component. have analysed in detail , there no memory leak in project. i not using strong property push viewcontroller. here how pushing viewcontroller. viewcontroller *obj = [[viewcontroller alloc] init]; [self.navigationcontroller pushviewcontroller:obj animated:no]; any clue whats going on? should free memory have consumed. please advise your description indicates have retain cycle, leads objects not being deallocated. typical source of retain cycles properties assigned loading nib files (usually declared iboutlet ). two strategies break them , release objects: declare property weak : @property (nonatomic, weak) iboutlet uilabel *statuslabel; set property nil in...
Read more

Django REST Framework perform_create: You cannot call `.save()` after accessing `serializer.data` -

-
i have following viewset: class artistprofileviewset(viewsets.modelviewset): queryset = artistprofile.objects.all() serializer_class = artistprofileserializer def perform_create(self, serializer): if serializer.is_valid(): serializer.save() with following serializers: class simpleartisttrackserializer(serializers.hyperlinkedmodelserializer): class meta: model = artisttrack fields = (...my fields...) class artistprofileserializer(serializers.hyperlinkedmodelserializer): owners = userserializer(many=true, required=false) tracks = simpleartisttrackserializer(many=true, required=false) class meta: model = artistprofile fields = (...my fields...) i getting following error: assertionerror: cannot call `.save()` after accessing `serializer.data`.if need access data before committing database inspect 'serializer.validated_data' instead. i don't see editing serializer.data object. th...
Read more

Why does Go error when trying to marshal this JSON? -

-
i'm trying rather simple example of trying decode json file structs using golang. however, when attempting error that json: cannot unmarshal object go value of type []main.track which don't understand. here's code in question, , json can found here . package main import ( "encoding/json" "fmt" "net/http" "os" ) // tracks group of track types type tracks struct { tracks []track } // track represents singular track type track struct { name string url string artist artist } // artist represents single artist type artist struct { name string mbid string url string } func main() { url := "http://ws.audioscrobbler.com/2.0/?method=geo.gettoptracks&api_key=c1572082105bd40d247836b5c1819623&format=json&country=netherlands" response, err := http.get(url) if err != nil { fmt.printf("%s\n", err) os.exit(1) } de...
Read more