LyX ⇒ "XeTeX is taking a long time" popup

[ExecutorElassus]

Hi Forum,

So, I've been digging around in the settings for Lyx, and recently reconfigured, and rebooted. In the meantime I've installed a few new packages (which I can go back and check for dependency updates if it comes to that), and now Lyx is doing something odd during compilation.

When I'm generating a preview to output to mupdf, a dialog box pops up and tells me that XeLaTeX is taking a long time, and asks if I want to let it keep running or stop it. Now, I don't have the longest document (223 pages at the moment), but it's fairly complex (text in Chinese, Hebrew, Russian, Arabic, excerpts from the Voynich Manuscript, a fairly large number of embedded PDFs, complicated footnotes and cross-references, etc. etc.) and compilation can sometimes take over 10 minutes.

Two questions: is there a way I can disable that dialog popup permanently? I don't want to have to go back to Lyx midway through compiling a document to confirm that yes, I know it's taking a long time, because it always takes a long time, and want it to continue.

Second, is this a normal compile time for a longer document? I was given to understand that it's common for tex to handle documents with thousands of figures and citations, hundreds of pages, etc. Is it usual for something like this to take >10 minutes on a modern system? Is there a way to speed it up?

Cheers,

EE

[Stefan Kottwitz]

Wow, that's long!

Perhaps tell us something about your document, so we can think about how to improve the speed. For example,

• Which packages do you use? (You can post the preamble with all \usepackage lines here). For example, TikZ can slow things down, or packages that built on TikZ and use it in the background, such as the todonotes package. TikZ is great, but requires resources.
• Do you generate images or just include them? You can externalize images, so, instead of creating them at each compile time again, they are created just once (as small pdf image) and included later only.

You can optimize compiling speed if you set a few options (somewhere in LyX settings). For example, you can get higher speed (but larger output, no compression!) by running XeLaTeX that way:

xelatex -interaction=nonstopmode -output-driver='xdvipdfmx -z0' filename.tex

z0 is zero compression, you can use z3 or z4 for usable compression or z9 for highest compression (the final compiling).

Stefan

[ExecutorElassus]

Hi Stefan,

thanks for your reply. Actually, this is a bit weirder than what I initially thought. Here's what's going on:

I build/install software from source (gentoo), so "installing updates" means a couple hours of all eight cores running at 100%. This is what I was doing when I was getting timeout popups. Now, with the system more or less idle, I don't get them.

So how is Lyx calculating whether xelatex is "taking a long time"? If xelatex is stuck waiting for CPU cycles because they're all being used by other processes, does that trigger a timeout popup?

On the second issue, of things taking a long time: the master document is a container for about 15 child documents. I've tried to strip all of them of any settings or preambles, so that only the stuff set at the beginning of master gets used. However, I have a fair number of modules: Initials, Custom Header/Footerlines, Biblatex-citation-styles, Logical Markup, Custom Paragraph Shapes, Multilingual Captions, and Customizable lists (enumitem). The preamble loads the following: csquotes, fontspec, xunicode, xcolor, xeCJK, polyglossia, xstring, bidi, bigfoot, footmisc, biblatex-chicago, setspace, chngcntr. And a bunch of custom polyglossia language definitions and custom font definitions.

The main text uses either .png or .pdf files embedded for figures, but some of them are large (greater than 8MB) files. There's also stuff going on with switching from western languages to (among others) ancient Greek, Chinese, Arabic, Hebrew, and Russian in the document, as well as doing weird stuff with footnotes (I use bigfoot to create a second layer of footnotes, to have footnotes of footnotes; I also use cross-references formatted superscript to have footnotes out of order). The bibliography is about 500 entries.

I'm sure all of this is resource-intensive. But it would be nice to cut compile time down a bit. Sometimes it compiles much faster (<1minute), so I'm wondering what's eating all the time with all the other compiles.

Thanks for the help!

EE

[Stefan Kottwitz]

Can you please post a .log file of the XeLaTeX run? That should be in the document folder. You can add it as attachment here. The attachment link is below the text edit field, when writing a post.

Stefan

[ExecutorElassus]

Hi Stefan,

I don't see a .log file for xetex specifically, but I do have a latex .log file in the temp directory Lyx uses to generate the pdf (the pdf, by the way, is 165MB). That log file is attached.

But if I'm supposed to be attaching some other log file, can you tell me where I should find it? It's not in the directory with all the .lyx files.

Cheers,

EE

[Stefan Kottwitz]

That's the correct .log file, there's only one.

I don't see any issues (some repeatedly included graphics, working with fonts may be slow). Well, if you noticed a quick compiling run, you could compare that new (quick) log file to the earlier one.

If you might see the actual compiling process in a progress window with output messages: do you notice where specifically it is slow?

Stefan

[scottkosty]

Hi Stefan,
I build/install software from source (gentoo), so "installing updates" means a couple hours of all eight cores running at 100%. This is what I was doing when I was getting timeout popups. Now, with the system more or less idle, I don't get them.

I don't believe there is an option in LyX to disable the popup. But I could probably give you a patch to the LyX code that would work, since you compile from source.

So how is Lyx calculating whether xelatex is "taking a long time"? If xelatex is stuck waiting for CPU cycles because they're all being used by other processes, does that trigger a timeout popup?

Definitely not that fancy. My guess is that it's a flat timer (which calculates time based on the wall clock). Note also that when LyX gives the popup, it doesn't pause the process, so I don't think any time is lost. There is only an annoyance of having to click on the dialog.

[ExecutorElassus]

Definitely not that fancy. My guess is that it's a flat timer (which calculates time based on the wall clock). Note also that when LyX gives the popup, it doesn't pause the process, so I don't think any time is lost. There is only an annoyance of having to click on the dialog.

But this is what's confusing me. I've only had that popup come up a couple days ago, when I was also compiling programs and running all the CPUs. Now, I have document compilations that take just as long (if not longer; it seemed like that popup was coming up every ~2-3 minutes), and I get no popup.

But I can't even reproduce it now, so I can't do much for it. I'll post more if it ever shows up again.

Cheers,

EE

[scottkosty]

Definitely not that fancy. My guess is that it's a flat timer (which calculates time based on the wall clock). Note also that when LyX gives the popup, it doesn't pause the process, so I don't think any time is lost. There is only an annoyance of having to click on the dialog.

But this is what's confusing me. I've only had that popup come up a couple days ago, when I was also compiling programs and running all the CPUs. Now, I have document compilations that take just as long (if not longer; it seemed like that popup was coming up every ~2-3 minutes), and I get no popup.

But I can't even reproduce it now, so I can't do much for it. I'll post more if it ever shows up again.

Cheers,

EE

Strange. Maybe I'm wrong then. If you can reproduce any strange behavior (especially if you can reproduce with LyX 2.3.0rc1 which will be available in October), then I would be interested in trying to figure out what's going on.

[ExecutorElassus]

My distro is still on 2.2.3, so I can't check against the RC yet. But now I timed it, and it's definitely not a flat timer. The popup comes up every three minutes if some other process is using all the CPUs (in my case, if I'm compiling in the background). But if the CPUs aren't all in use by some other program, document compilation takes way longer than three minutes with no popup.

Is there some threshold number of "completed steps per unit time" that triggers that warning? Is this timer also supposed to catch compilations if the compilation hangs (such that active compilation stops, but doesn't exit, getting stuck), and not just measure/respond to how long the job is taking?

Cheers,

EE