[fess-user 81] Re: クロール中のはずが3時間ほど何も起こらないのですが...

Back to archive index

Masayuki Shibata mshib****@shima*****
2010年 1月 7日 (木) 16:04:28 JST


Core2 Duo E8500 3.16GHz 2GB RAM
Windows Xp マシンです。

タスクマネージャで見ると java.exe のメモリ使用量は 650,000K
確保された仮想メモリサイズは 672,000K ほどです。

上記はクロール中ですので、起動直後だとそれぞれ 127,200K、
171,600K ほどです。このとき物理メモリの空きは 1GB ほどありま
す。


>菅谷です。
>
>試していただきありがとうございます。
>
>ClientAbortException に関しては、おそらく管理画面を
>表示していて、読み込みを終了前に次のリクエストが
>発生してしまったため、発生したものと思われます。
>(リロードを連続で呼ぶとそれが出たりすると思います)
>無視してしまって問題ないので、これを拾わないように
>します。
>
>ですので、その Exception はクロール処理に影響は
>ない感じですが、GC でないとすると、Solr のドキュメントも
>秒レベルの時間がかかることから、マシンスペック的に
>それくらいの可能性も考えられます。クロール処理は
>そこそこメモリーを使うので(条件によりますが観測して
>いる感じですと、200m〜300mとか)、これらがスワップに
>飛ばされたりすると遅くなると思います。ということも
>あるのですが、マシンスペック的にはいかがでしょうか?
>
>shinsuke
>
>
>2010年1月7日14:59 Masayuki Shibata <mshib****@shima*****>:
>> 柴田@亀岡市です。
>>
>> メモリ確保してやってみてますが、あまり速くはなっていないようです。
>>
>> 以下のログですと 2010-01-07 13:47:23,578、2010-01-07 14:03:01,921
>> の間 16 分空いてます。
>>
>> その後今 (14:55) までずっと応答がないのでログを開いて見たら、例外
>> が出ておりましたので以下に貼り付けておきます。
>>
>> 16 分ですと、以前の 1 時間とか 40 分とかと比較して速いのか遅いの
>> かわからないのですが、その後の
>>        Indexing http://hostname/svn/pjname/.../.*
>> 行が登場する前にエラーが出てしまいましたし、現時点で 50 分以上経過
>> していますので、ここでいったん止めます。
>>
>>
>> 2010-01-07 13:47:23,578 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Indexing http://hostname/svn/pjname/.../filename5.log
>> 2010-01-07 13:53:52,750 [http-8080-2] INFO
>> org.apache.commons.httpclient.auth.AuthChallengeProcessor - basic authentication scheme 
selected
>> 2010-01-07 14:03:01,921 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Set a document 
boost
>> (100.0).
>> 2010-01-07 14:03:01,921 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Added the 
document.
>> The number of a document cache is 8.
>> 2010-01-07 14:03:01,921 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - The number of an
>> added document is 8.
>> 2010-01-07 14:03:01,921 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Indexing http://hostname/svn/pjname/.../filename3.log
>> 2010-01-07 14:06:02,453 [http-8080-2] INFO
>> org.apache.commons.httpclient.auth.AuthChallengeProcessor - basic authentication scheme 
selected
>> 2010-01-07 14:23:27,500 [http-8080-2] INFO
>> org.apache.commons.httpclient.auth.AuthChallengeProcessor - basic authentication scheme 
selected
>> 2010-01-07 14:25:12,750 [http-8080-2] ERROR
>> jp.sf.fess.interceptor.FessActionMessagesThrowsInterceptor - System error occured.
>> org.seasar.framework.exception.IORuntimeException: [ESSR0040]IO例外が発生しました。理由は
>> ClientAbortException:  java.net.SocketException: Connection reset by peer: socket write error
>>        at org.seasar.framework.util.InputStreamUtil.copy(InputStreamUtil.java:98)
>>        at org.seasar.struts.util.ResponseUtil.download(ResponseUtil.java:93)
>>        at jp.sf.fess.action.admin.LogAction.download(LogAction.java:82)
>>        at jp.sf.fess.action.admin.LogAction$$EnhancedByS2AOP$$818805.$$download
>> $$invokeSuperMethod$$(LogAction$$EnhancedByS2AOP$$818805.java)
>>        at jp.sf.fess.action.admin.LogAction$$EnhancedByS2AOP$$818805$$MethodInvocation
>> $$download0.proceed(MethodInvocationClassGenerator.java)
>>        at org.seasar.framework.aop.interceptors.ThrowsInterceptor.invoke
>> (ThrowsInterceptor.java:79)
>>        at jp.sf.fess.interceptor.FessActionMessagesThrowsInterceptor.invoke
>> (FessActionMessagesThrowsInterceptor.java:36)
>>        at jp.sf.fess.action.admin.LogAction$$EnhancedByS2AOP$$818805$$MethodInvocation
>> $$download0.proceed(MethodInvocationClassGenerator.java)
>>        at org.seasar.framework.aop.interceptors.TraceInterceptor.invoke
>> (TraceInterceptor.java:56)
>>        at jp.sf.fess.action.admin.LogAction$$EnhancedByS2AOP$$818805$$MethodInvocation
>> $$download0.proceed(MethodInvocationClassGenerator.java)
>>        at jp.sf.fess.action.admin.LogAction$$EnhancedByS2AOP$$818805.download(LogAction
>> $$EnhancedByS2AOP$$818805.java)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>        at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:25)
>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>        at org.seasar.framework.util.MethodUtil.invoke(MethodUtil.java:96)
>>        at org.seasar.struts.action.ActionWrapper.execute(ActionWrapper.java:139)
>>        at org.seasar.struts.action.ActionWrapper.execute(ActionWrapper.java:87)
>>        at org.apache.struts.action.RequestProcessor.processActionPerform
>> (RequestProcessor.java:431)
>>        at org.seasar.struts.action.S2RequestProcessor.process(S2RequestProcessor.java:132)
>>        at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196)
>>        at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:414)
>>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
>>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>>        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
>> (ApplicationFilterChain.java:290)
>>        at org.apache.catalina.core.ApplicationFilterChain.doFilter
>> (ApplicationFilterChain.java:206)
>>        at org.seasar.framework.container.hotdeploy.HotdeployFilter.doFilter
>> (HotdeployFilter.java:71)
>>        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
>> (ApplicationFilterChain.java:235)
>>        at org.apache.catalina.core.ApplicationFilterChain.doFilter
>> (ApplicationFilterChain.java:206)
>>        at org.seasar.framework.container.filter.S2ContainerFilter.doFilter
>> (S2ContainerFilter.java:79)
>>        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
>> (ApplicationFilterChain.java:235)
>>        at org.apache.catalina.core.ApplicationFilterChain.doFilter
>> (ApplicationFilterChain.java:206)
>>        at org.mobylet.core.http.MobyletFilter.processFilter(MobyletFilter.java:103)
>>        at org.mobylet.core.http.MobyletFilter.doFilter(MobyletFilter.java:67)
>>        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
>> (ApplicationFilterChain.java:235)
>>        at org.apache.catalina.core.ApplicationFilterChain.doFilter
>> (ApplicationFilterChain.java:206)
>>        at org.apache.catalina.core.ApplicationDispatcher.invoke
(ApplicationDispatcher.java:646)
>>        at org.apache.catalina.core.ApplicationDispatcher.processRequest
>> (ApplicationDispatcher.java:436)
>>        at org.apache.catalina.core.ApplicationDispatcher.doForward
>> (ApplicationDispatcher.java:374)
>>        at org.apache.catalina.core.ApplicationDispatcher.forward
(ApplicationDispatcher.java:302)
>>        at org.seasar.struts.filter.RoutingFilter.forward(RoutingFilter.java:219)
>>        at org.seasar.struts.filter.RoutingFilter.doFilter(RoutingFilter.java:99)
>>        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
>> (ApplicationFilterChain.java:235)
>>        at org.apache.catalina.core.ApplicationFilterChain.doFilter
>> (ApplicationFilterChain.java:206)
>>        at jp.sf.fess.filter.AuthenticationFilter.doFilter(AuthenticationFilter.java:98)
>>        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
>> (ApplicationFilterChain.java:235)
>>        at org.apache.catalina.core.ApplicationFilterChain.doFilter
>> (ApplicationFilterChain.java:206)
>>        at org.seasar.framework.container.hotdeploy.HotdeployFilter.doFilter
>> (HotdeployFilter.java:71)
>>        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
>> (ApplicationFilterChain.java:235)
>>        at org.apache.catalina.core.ApplicationFilterChain.doFilter
>> (ApplicationFilterChain.java:206)
>>        at org.seasar.framework.container.filter.S2ContainerFilter.doFilter
>> (S2ContainerFilter.java:79)
>>        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
>> (ApplicationFilterChain.java:235)
>>        at org.apache.catalina.core.ApplicationFilterChain.doFilter
>> (ApplicationFilterChain.java:206)
>>        at org.mobylet.core.http.MobyletFilter.processFilter(MobyletFilter.java:103)
>>        at org.mobylet.core.http.MobyletFilter.doFilter(MobyletFilter.java:67)
>>        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
>> (ApplicationFilterChain.java:235)
>>        at org.apache.catalina.core.ApplicationFilterChain.doFilter
>> (ApplicationFilterChain.java:206)
>>        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
>>        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
>>        at org.apache.catalina.authenticator.AuthenticatorBase.invoke
(AuthenticatorBase.java:433)
>>        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
>>        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>>        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>>        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
>>        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
>>        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process
>> (Http11Protocol.java:583)
>>        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
>>        at java.lang.Thread.run(Thread.java:619)
>> Caused by: ClientAbortException:  java.net.SocketException: Connection reset by peer: socket
>> write error
>>        at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:358)
>>        at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:325)
>>        at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:381)
>>        at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:370)
>>        at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:89)
>>        at org.seasar.framework.util.InputStreamUtil.copy(InputStreamUtil.java:95)
>>        ... 67 more
>> Caused by: java.net.SocketException: Connection reset by peer: socket write error
>>        at java.net.SocketOutputStream.socketWrite0(Native Method)
>>        at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
>>        at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
>>        at org.apache.coyote.http11.InternalOutputBuffer.realWriteBytes
>> (InternalOutputBuffer.java:740)
>>        at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:434)
>>        at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:349)
>>        at org.apache.coyote.http11.InternalOutputBuffer$OutputStreamOutputBuffer.doWrite
>> (InternalOutputBuffer.java:764)
>>        at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite
>> (ChunkedOutputFilter.java:126)
>>        at org.apache.coyote.http11.InternalOutputBuffer.doWrite(InternalOutputBuffer.java:573)
>>        at org.apache.coyote.Response.doWrite(Response.java:560)
>>        at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:353)
>>        ... 72 more
>> 2010-01-07 14:25:12,750 [http-8080-3] ERROR
>> jp.sf.fess.interceptor.FessActionMessagesThrowsInterceptor - System error occured.
>> org.seasar.framework.exception.IORuntimeException: [ESSR0040]IO例外が発生しました。理由は
>> ClientAbortException:  java.net.SocketException: Connection reset by peer: socket write error
>>        at org.seasar.framework.util.InputStreamUtil.copy(InputStreamUtil.java:98)
>>        at org.seasar.struts.util.ResponseUtil.download(ResponseUtil.java:93)
>>        at jp.sf.fess.action.admin.LogAction.download(LogAction.java:82)
>>        at jp.sf.fess.action.admin.LogAction$$EnhancedByS2AOP$$818805.$$download
>> $$invokeSuperMethod$$(LogAction$$EnhancedByS2AOP$$818805.java)
>>        at jp.sf.fess.action.admin.LogAction$$EnhancedByS2AOP$$818805$$MethodInvocation
>> $$download0.proceed(MethodInvocationClassGenerator.java)
>>        at org.seasar.framework.aop.interceptors.ThrowsInterceptor.invoke
>> (ThrowsInterceptor.java:79)
>>        at jp.sf.fess.interceptor.FessActionMessagesThrowsInterceptor.invoke
>> (FessActionMessagesThrowsInterceptor.java:36)
>>        at jp.sf.fess.action.admin.LogAction$$EnhancedByS2AOP$$818805$$MethodInvocation
>> $$download0.proceed(MethodInvocationClassGenerator.java)
>>        at org.seasar.framework.aop.interceptors.TraceInterceptor.invoke
>> (TraceInterceptor.java:56)
>>        at jp.sf.fess.action.admin.LogAction$$EnhancedByS2AOP$$818805$$MethodInvocation
>> $$download0.proceed(MethodInvocationClassGenerator.java)
>>        at jp.sf.fess.action.admin.LogAction$$EnhancedByS2AOP$$818805.download(LogAction
>> $$EnhancedByS2AOP$$818805.java)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>        at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:25)
>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>        at org.seasar.framework.util.MethodUtil.invoke(MethodUtil.java:96)
>>        at org.seasar.struts.action.ActionWrapper.execute(ActionWrapper.java:139)
>>        at org.seasar.struts.action.ActionWrapper.execute(ActionWrapper.java:87)
>>        at org.apache.struts.action.RequestProcessor.processActionPerform
>> (RequestProcessor.java:431)
>>        at org.seasar.struts.action.S2RequestProcessor.process(S2RequestProcessor.java:132)
>>        at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196)
>>        at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:414)
>>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
>>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
>>        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
>> (ApplicationFilterChain.java:290)
>>        at org.apache.catalina.core.ApplicationFilterChain.doFilter
>> (ApplicationFilterChain.java:206)
>>        at org.seasar.framework.container.hotdeploy.HotdeployFilter.doFilter
>> (HotdeployFilter.java:71)
>>        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
>> (ApplicationFilterChain.java:235)
>>        at org.apache.catalina.core.ApplicationFilterChain.doFilter
>> (ApplicationFilterChain.java:206)
>>        at org.seasar.framework.container.filter.S2ContainerFilter.doFilter
>> (S2ContainerFilter.java:79)
>>        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
>> (ApplicationFilterChain.java:235)
>>        at org.apache.catalina.core.ApplicationFilterChain.doFilter
>> (ApplicationFilterChain.java:206)
>>        at org.mobylet.core.http.MobyletFilter.processFilter(MobyletFilter.java:103)
>>        at org.mobylet.core.http.MobyletFilter.doFilter(MobyletFilter.java:67)
>>        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
>> (ApplicationFilterChain.java:235)
>>        at org.apache.catalina.core.ApplicationFilterChain.doFilter
>> (ApplicationFilterChain.java:206)
>>        at org.apache.catalina.core.ApplicationDispatcher.invoke
(ApplicationDispatcher.java:646)
>>        at org.apache.catalina.core.ApplicationDispatcher.processRequest
>> (ApplicationDispatcher.java:436)
>>        at org.apache.catalina.core.ApplicationDispatcher.doForward
>> (ApplicationDispatcher.java:374)
>>        at org.apache.catalina.core.ApplicationDispatcher.forward
(ApplicationDispatcher.java:302)
>>        at org.seasar.struts.filter.RoutingFilter.forward(RoutingFilter.java:219)
>>        at org.seasar.struts.filter.RoutingFilter.doFilter(RoutingFilter.java:99)
>>        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
>> (ApplicationFilterChain.java:235)
>>        at org.apache.catalina.core.ApplicationFilterChain.doFilter
>> (ApplicationFilterChain.java:206)
>>        at jp.sf.fess.filter.AuthenticationFilter.doFilter(AuthenticationFilter.java:98)
>>        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
>> (ApplicationFilterChain.java:235)
>>        at org.apache.catalina.core.ApplicationFilterChain.doFilter
>> (ApplicationFilterChain.java:206)
>>        at org.seasar.framework.container.hotdeploy.HotdeployFilter.doFilter
>> (HotdeployFilter.java:71)
>>        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
>> (ApplicationFilterChain.java:235)
>>        at org.apache.catalina.core.ApplicationFilterChain.doFilter
>> (ApplicationFilterChain.java:206)
>>        at org.seasar.framework.container.filter.S2ContainerFilter.doFilter
>> (S2ContainerFilter.java:79)
>>        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
>> (ApplicationFilterChain.java:235)
>>        at org.apache.catalina.core.ApplicationFilterChain.doFilter
>> (ApplicationFilterChain.java:206)
>>        at org.mobylet.core.http.MobyletFilter.processFilter(MobyletFilter.java:103)
>>        at org.mobylet.core.http.MobyletFilter.doFilter(MobyletFilter.java:67)
>>        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter
>> (ApplicationFilterChain.java:235)
>>        at org.apache.catalina.core.ApplicationFilterChain.doFilter
>> (ApplicationFilterChain.java:206)
>>        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
>>        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
>>        at org.apache.catalina.authenticator.AuthenticatorBase.invoke
(AuthenticatorBase.java:433)
>>        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
>>        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>>        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>>        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
>>        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
>>        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process
>> (Http11Protocol.java:583)
>>        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
>>        at java.lang.Thread.run(Thread.java:619)
>> Caused by: ClientAbortException:  java.net.SocketException: Connection reset by peer: socket
>> write error
>>        at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:358)
>>        at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:325)
>>        at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:381)
>>        at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:370)
>>        at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:89)
>>        at org.seasar.framework.util.InputStreamUtil.copy(InputStreamUtil.java:95)
>>        ... 67 more
>> Caused by: java.net.SocketException: Connection reset by peer: socket write error
>>        at java.net.SocketOutputStream.socketWrite0(Native Method)
>>        at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
>>        at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
>>        at org.apache.coyote.http11.InternalOutputBuffer.realWriteBytes
>> (InternalOutputBuffer.java:740)
>>        at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:434)
>>        at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:349)
>>        at org.apache.coyote.http11.InternalOutputBuffer$OutputStreamOutputBuffer.doWrite
>> (InternalOutputBuffer.java:764)
>>        at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite
>> (ChunkedOutputFilter.java:126)
>>        at org.apache.coyote.http11.InternalOutputBuffer.doWrite(InternalOutputBuffer.java:573)
>>        at org.apache.coyote.Response.doWrite(Response.java:560)
>>        at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:353)
>>        ... 72 more
>>
>>>菅谷です。
>>>
>>>調査していただきありがとうございます。
>>>
>>>私もここのところ、厳しめのテストをいろいろと
>>>実行していますが、1M近くサイズがあることと
>>>IndexUpdater が普通の処理で止まっている
>>>ことを考えると、GC で時間がかかっている
>>>ことも考えられます。ヒープがいっぱいになると、
>>>GC で確保しようとして、IndexUpdater とかで
>>>停止しているような状態を観測したことがあります。
>>>ですので、bin/setenv.[bat|sh] の -Xmx512m を
>>>-Xmx1024m とか変更すると改善するかも
>>>しれません。
>>>
>>>> Found a TextHeaderAtom not followed by a TextBytesAtom or TextCharsAtom: Followed by 3999
>>>
>>>上記については、POI が出しているようなので
>>>何とかできるものなのか、時間があるときに
>>>見てみます。
>>>
>>>shinsuke
>>>
>>>
>>>2010年1月6日17:55 Masayuki Shibata <mshib****@shima*****>:
>>>> 柴田@亀岡市です。
>>>>
>>>> さらに訂正です。バタバタしてスミマセン。
>>>>
>>>>>ウソ書いてしまいました。ちゃんと終わっていないようです。
>>>> ...
>>>>>...と、この種のファイルに出会うと「[EXEC TIME] crawling time: xxxxxms」が
>>>>>出ます。
>>>>>
>>>>>document cache が 11 に到達する前にクロールを止めてしまっているようで、
>>>>>その後の...
>>>>>[EXEC TIME] index update time: xxxxxms
>>>>>[EXEC TIME] index optimize time: xxxxxms
>>>>>...が登場せず、クローラープロセスが終了しません。
>>>>
>>>> 終了していないのではなく、今まで同様にファイルのインデックス作るのが
>>>> 超遅くて、次のログ行が出てくるまでに時間間隔があいていただけでした。
>>>>
>>>> 以下、そのときのログです。
>>>>
>>>> 問題のファイルは 拡張子 log のもので、同じフォルダ配下に 拡張子 c の
>>>> ものも少しあり、それはすぐ終わっていますが、log には時間を要している
>>>> という図式です。
>>>>
>>>> ...
>>>> 2010-01-06 17:23:02,234 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Processing
>> documents
>>>> in IndexUpdater queue.
>>>> 2010-01-06 17:23:02,687 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - The number of a
>>>> crawled document is 12. The processing size is 10. The execution time is 453ms.
>>>> 2010-01-06 17:23:02,687 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Indexing http://.../
>>>> 2010-01-06 17:23:02,687 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Set a document
>> boost
>>>> (100.0).
>>>> 2010-01-06 17:23:02,687 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Added the
>> document.
>>>> The number of a document cache is 1.
>>>> 2010-01-06 17:23:02,687 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - The number of an
>>>> added document is 1.
>>>> 2010-01-06 17:23:02,687 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Indexing http://.../filename1.c
>>>> 2010-01-06 17:23:02,687 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Set a document
>> boost
>>>> (100.0).
>>>> 2010-01-06 17:23:02,687 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Added the
>> document.
>>>> The number of a document cache is 2.
>>>> 2010-01-06 17:23:02,687 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - The number of an
>>>> added document is 2.
>>>> 2010-01-06 17:23:02,687 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Indexing http://.../filename2.c
>>>> 2010-01-06 17:23:02,921 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Set a document
>> boost
>>>> (100.0).
>>>> 2010-01-06 17:23:02,921 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Added the
>> document.
>>>> The number of a document cache is 3.
>>>> 2010-01-06 17:23:02,921 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - The number of an
>>>> added document is 3.
>>>> 2010-01-06 17:23:02,921 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Indexing http://.../filename3.c
>>>> 2010-01-06 17:23:03,125 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Set a document
>> boost
>>>> (100.0).
>>>> 2010-01-06 17:23:03,125 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Added the
>> document.
>>>> The number of a document cache is 4.
>>>> 2010-01-06 17:23:03,125 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - The number of an
>>>> added document is 4.
>>>> 2010-01-06 17:23:03,125 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Indexing http://.../filename4.c
>>>> 2010-01-06 17:23:03,375 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Set a document
>> boost
>>>> (100.0).
>>>> 2010-01-06 17:23:03,375 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Added the
>> document.
>>>> The number of a document cache is 5.
>>>> 2010-01-06 17:23:03,375 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - The number of an
>>>> added document is 5.
>>>> 2010-01-06 17:23:03,375 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Indexing http://.../filename5.c
>>>> 2010-01-06 17:23:03,609 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Set a document
>> boost
>>>> (100.0).
>>>> 2010-01-06 17:23:03,609 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Added the
>> document.
>>>> The number of a document cache is 6.
>>>> 2010-01-06 17:23:03,609 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - The number of an
>>>> added document is 6.
>>>> 2010-01-06 17:23:03,609 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Indexing http://.../filename1.log
>>>> 2010-01-06 17:23:21,375 [http-8080-3] INFO
>>>> org.apache.commons.httpclient.auth.AuthChallengeProcessor - basic authentication scheme
>> selected
>>>> 2010-01-06 17:23:31,453 [Thread-22] INFO  jp.sf.fess.helper.WebIndexHelper - [EXEC TIME]
>> crawling
>>>> time: 89219ms
>>>> 2010-01-06 17:36:02,218 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Set a document
>> boost
>>>> (100.0).
>>>> 2010-01-06 17:36:02,218 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Added the
>> document.
>>>> The number of a document cache is 7.
>>>> 2010-01-06 17:36:02,218 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - The number of an
>>>> added document is 7.
>>>> 2010-01-06 17:36:02,218 [IndexUpdater] DEBUG jp.sf.fess.solr.IndexUpdater - Indexing http://.../filename2.log
>>>> 2010-01-06 17:39:54,218 [http-8080-3] INFO
>>>> org.apache.commons.httpclient.auth.AuthChallengeProcessor - basic authentication scheme
>> selected
>>>>
>>>> _______________________________________________
>>>> Fess-user mailing list
>>>> Fess-****@lists*****
>>>> http://lists.sourceforge.jp/mailman/listinfo/fess-user
>>>>
>>>
>>>_______________________________________________
>>>Fess-user mailing list
>>>Fess-****@lists*****
>>>http://lists.sourceforge.jp/mailman/listinfo/fess-user
>>>
>>
>> ----------
>> Shibata, Masayuki
>> Manager, Software Development
>> Shimadzu Corporation
>> Analytical Measuring Instruments Division
>> Research & Development Department
>> Telephone:  075-823-1441 Japan
>> Facsimile:  075-823-1365 Japan
>> E-mail: mshib****@shima*****
>>
>> _______________________________________________
>> Fess-user mailing list
>> Fess-****@lists*****
>> http://lists.sourceforge.jp/mailman/listinfo/fess-user
>>
>
>_______________________________________________
>Fess-user mailing list
>Fess-****@lists*****
>http://lists.sourceforge.jp/mailman/listinfo/fess-user
>

----------
Shibata, Masayuki
Manager, Software Development
Shimadzu Corporation
Analytical Measuring Instruments Division
Research & Development Department
Telephone:  075-823-1441 Japan
Facsimile:  075-823-1365 Japan
E-mail: mshib****@shima*****




Fess-user メーリングリストの案内
Back to archive index