Pandas Input/output error? -

-

i created multiindex .h5 file , use hdfstore store data. however, when try read dataframe file, turns out can read dataframes fail read several others. following error message get:

hdf5-diag: error detected in hdf5 (1.8.9) thread 0: #000: h5dio.c line 174 in h5dread(): can't read data major: dataset minor: read failed

#001: h5dio.c line 449 in h5d_read(): can't read data major: dataset minor: read failed

#002: h5dcontig.c line 560 in h5d_contig_read(): contiguous read failed major: dataset minor: read failed

#003: h5dselect.c line 275 in h5d_select_read(): read error major: dataspace minor: read failed

#004: h5dselect.c line 210 in h5d_select_io(): read error major: dataspace minor: read failed

#005: h5dcontig.c line 890 in h5d_contig_readvv(): can't perform vectorized sieve buffer read major: dataset minor: can't operate on object

#006: h5v.c line 1457 in h5v_opvv(): can't perform operation major: internal error (too specific document in detail) minor: can't operate on object

#007: h5dcontig.c line 691 in h5d_contig_readvv_sieve_cb(): block read failed major: dataset minor: read failed

#008: h5fio.c line 113 in h5f_block_read(): read through metadata accumulator failed major: low-level i/o minor: read failed

#009: h5faccum.c line 254 in h5f_accum_read(): driver read request failed major: low-level i/o minor: read failed

#010: h5fdint.c line 142 in h5fd_read(): driver read request failed major: virtual file layer minor: read failed

#011: h5fdsec2.c line 753 in h5fd_sec2_read(): file read failed: time = wed jul 3 14:07:31 2013

, filename = 'n.h5', file descriptor = 3, errno = 5, error message = 'input/output error', buf = 0x7f8807fccb6c, size = 649492, offset = 185876480

major: low-level i/o  minor: read failed

traceback (most recent call last):

file "", line 1, in

file "/usr/share/python27/lib/python2.7/site-packages/pandas/io/pytables.py", line 165, in getitem

return self.get(key)

file "/usr/share/python27/lib/python2.7/site-packages/pandas/io/pytables.py", line 261, in get

return self._read_group(group)

file "/usr/share/python27/lib/python2.7/site-packages/pandas/io/pytables.py", line 651, in _read_group return handler(group, where)

file "/usr/share/python27/lib/python2.7/site-packages/pandas/io/pytables.py", line 400, in _read_frame return dataframe(self._read_block_manager(group))

file "/usr/share/python27/lib/python2.7/site-packages/pandas/io/pytables.py", line 426, in _read_block_manager ax = self._read_index(group, 'axis%d' % i)

file "/usr/share/python27/lib/python2.7/site-packages/pandas/io/pytables.py", line 487, in _read_index _, index = self._read_index_node(getattr(group, key))

file "/usr/share/python27/lib/python2.7/site-packages/pandas/io/pytables.py", line 532, in _read_index_node data = node[:]

file "/usr/share/python27/lib/python2.7/site-packages/tables/array.py", line 684, in getitem arr = self._readslice(startl, stopl, stepl, shape)

file "/usr/share/python27/lib/python2.7/site-packages/tables/array.py", line 781, in _readslice self._g_readslice(startl, stopl, stepl, nparr)

file "hdf5extension.pyx", line 1092, in tables.hdf5extension.array._g_readslice (tables/hdf5extension.c:9570) tables.exceptions.hdf5exterror: problems reading array data.

can problem ? thank much.

the hdf5 file corrupt. can occur doing write, interrupting in middle (ctrl-c). unable recover. hdf5 writes in chunks , if chunk interrupted believe state not consistent.

you corruption if doing concurrent writes (either threads or processes). see this: http://pandas.pydata.org/pandas-docs/dev/io.html#notes-caveats


Comments

Post a Comment

Popular posts from this blog

Django REST Framework perform_create: You cannot call `.save()` after accessing `serializer.data` -

-
i have following viewset: class artistprofileviewset(viewsets.modelviewset): queryset = artistprofile.objects.all() serializer_class = artistprofileserializer def perform_create(self, serializer): if serializer.is_valid(): serializer.save() with following serializers: class simpleartisttrackserializer(serializers.hyperlinkedmodelserializer): class meta: model = artisttrack fields = (...my fields...) class artistprofileserializer(serializers.hyperlinkedmodelserializer): owners = userserializer(many=true, required=false) tracks = simpleartisttrackserializer(many=true, required=false) class meta: model = artistprofile fields = (...my fields...) i getting following error: assertionerror: cannot call `.save()` after accessing `serializer.data`.if need access data before committing database inspect 'serializer.validated_data' instead. i don't see editing serializer.data object. th...
Read more

Why does Go error when trying to marshal this JSON? -

-
i'm trying rather simple example of trying decode json file structs using golang. however, when attempting error that json: cannot unmarshal object go value of type []main.track which don't understand. here's code in question, , json can found here . package main import ( "encoding/json" "fmt" "net/http" "os" ) // tracks group of track types type tracks struct { tracks []track } // track represents singular track type track struct { name string url string artist artist } // artist represents single artist type artist struct { name string mbid string url string } func main() { url := "http://ws.audioscrobbler.com/2.0/?method=geo.gettoptracks&api_key=c1572082105bd40d247836b5c1819623&format=json&country=netherlands" response, err := http.get(url) if err != nil { fmt.printf("%s\n", err) os.exit(1) } de...
Read more